Tags: #inference-framework
AI Inference Framework
Python
25.7k
sgl-project/sglang
A high-performance serving framework designed to accelerate inference for large language models and multimodal AI models.
Multimodal AI Inference and Serving Framework
python
4.4k
vllm-project/vllm-omni
vLLM-Omni is an efficient, flexible, and easy-to-use framework extending vLLM to serve omni-modality models (text, image, video, audio) with high throughput and an OpenAI-compatible API.