OSS Alternative - Discover Top Open Source Alternatives to Popular Software

ymcui/Chinese-LLaMA-Alpaca-2

An open-source project providing Chinese LLaMA-2 and Alpaca-2 large language models with enhanced Chinese capabilities and support for ultra-long contexts up to 64K.

Core Features

Open-source Chinese LLaMA-2 base models and Alpaca-2 instruction-tuned models with expanded Chinese vocabulary.

Support for 16K and 64K ultra-long context lengths.

Includes RLHF-aligned models for improved value alignment.

Provides training and fine-tuning scripts for custom model development.

Compatible with popular LLM ecosystem tools like Hugging Face Transformers, llama.cpp, and LangChain.

Detailed Introduction

This project is the second phase of the Chinese LLaMA & Alpaca initiative, building upon Meta's commercially viable Llama-2. It offers Chinese LLaMA-2 base models and Alpaca-2 instruction-tuned models, significantly enhancing Chinese semantic understanding and instruction following by expanding the Chinese vocabulary and pre-training with extensive Chinese data. The project supports FlashAttention-2 for efficient training and provides models with standard 4K, and extended 16K/64K context lengths, alongside RLHF-aligned versions for better ethical performance. It aims to empower developers with robust, open-source Chinese LLMs.