Tags: #llm-serving

SGLang is a high-performance serving framework for large language models and multimodal models, optimizing inference throughput and latency.

A KVCache-centric disaggregated architecture for high-performance LLM serving, powering leading AI services.

A framework for easily self-hosting and serving any open-source Large Language Models as OpenAI-compatible API endpoints in the cloud.

A cloud-native framework for building and deploying high-performance multimodal AI applications with built-in scaling and orchestration.