Tags: #gpu-optimization

A unified, open platform for accelerating AI model serving and scaling GenAI deployments with industry-leading performance across various hardware.

Firefly is an open-source, all-in-one tool designed for efficient pre-training, instruction fine-tuning, and DPO of a wide range of mainstream large language models, optimized for resource-constrained environments.

Optimizes large language model inference to run 70B models on a single 4GB GPU without quantization, enabling efficient deployment on resource-constrained hardware.

A multi-LoRA inference server designed to serve thousands of fine-tuned LLMs on a single GPU, significantly reducing serving costs while maintaining high throughput and low latency.