OSS Alternative - Discover Top Open Source Alternatives to Popular Software

ymcui/Chinese-LLaMA-Alpaca-2

An open-source project providing Chinese LLaMA-2 and Alpaca-2 large language models with expanded Chinese vocabulary, enhanced capabilities, and support for ultra-long contexts up to 64K.

Core Features

Open-source Chinese LLaMA-2 base models and Alpaca-2 instruction-tuned models.

Expanded and optimized Chinese vocabulary for Llama-2.

Support for 16K and 64K ultra-long context lengths.

Includes RLHF models for human preference alignment and improved value embodiment.

Provides pre-training and instruction-tuning scripts for custom model development.

Supports local deployment and quantization on CPU/GPU, integrating with popular LLM ecosystem tools.

Detailed Introduction

This project, Chinese-LLaMA-Alpaca-2, is the second iteration of the Chinese LLaMA & Alpaca initiative, building upon Meta's commercially viable Llama-2. It offers open-source Chinese LLaMA-2 base models and Alpaca-2 instruction-tuned models. Key enhancements include an expanded and optimized Chinese vocabulary, extensive incremental pre-training with large-scale Chinese data, significantly boosting Chinese semantic understanding and instruction following. The project features models supporting standard 4K context, as well as long-context versions up to 64K, and RLHF-aligned models for better value representation. It also provides training scripts and supports various LLM ecosystem tools for flexible deployment and further development.