AI Inference Engine & Optimization Library
3.8k 2026-04-18
nunchaku-ai/nunchaku
Nunchaku is a high-performance inference engine that optimizes 4-bit neural networks, especially diffusion models, for speed and efficiency.
Core Features
High-performance 4-bit neural network inference.
Significant VRAM reduction through asynchronous offloading.
Seamless integration with ComfyUI and LoRA support.
Broad GPU compatibility, including INT4 for 20-series GPUs.
Optimized for popular diffusion models like Qwen-Image and Z-Image.
Detailed Introduction
Nunchaku is an advanced, high-performance inference engine built upon the SVDQuant paper, designed to accelerate 4-bit neural networks. It focuses on optimizing diffusion models by leveraging low-rank components to absorb outliers, enabling efficient 4-bit quantization. The project significantly boosts inference speed, reduces VRAM consumption to as little as 3 GiB, and offers robust integration with platforms like ComfyUI, making it a powerful tool for deploying efficient AI models.