Deep Learning Library
8.2k 2026-04-30
bitsandbytes-foundation/bitsandbytes
A PyTorch library enabling accessible large language models through k-bit quantization, significantly reducing memory consumption for both inference and training.
Core Features
8-bit optimizers for memory-efficient training with 32-bit performance.
LLM.int8() for 8-bit large language model inference with half memory and no performance degradation.
QLoRA (4-bit quantization) for memory-saving LLM training without performance compromise.
Provides quantization primitives for 8-bit and 4-bit operations.
Quick Start
pip install bitsandbytesDetailed Introduction
Bitsandbytes is a crucial PyTorch library designed to make large language models more accessible by dramatically reducing their memory footprint. It achieves this through advanced k-bit quantization techniques, including 8-bit optimizers for efficient training, LLM.int8() for high-performance 8-bit inference, and QLoRA for 4-bit training without compromising model quality. This enables developers to run and train larger models on more modest hardware, democratizing access to cutting-edge AI.