Tags: #gpu-optimization

A unified, open platform for accelerating AI model serving and scaling GenAI deployments with industry-leading performance across various hardware.

Firefly is an open-source toolkit for efficient large language model training, supporting pre-training, instruction fine-tuning, and DPO with methods like LoRA and QLoRA.

AirLLM optimizes large language model inference memory, enabling 70B LLMs on a single 4GB GPU without quantization, and 405B Llama3.1 on 8GB VRAM.

A multi-LoRA inference server designed to efficiently serve thousands of fine-tuned Large Language Models on a single GPU, drastically cutting serving costs while maintaining high throughput and low latency.