Machine Learning Compiler and LLM Deployment Engine
22.5k 2026-04-20
mlc-ai/mlc-llm
A universal machine learning compiler and high-performance deployment engine for large language models, enabling native execution across diverse platforms.
Core Features
Universal LLM deployment across various platforms (AMD, NVIDIA, Apple, Intel GPUs, Web, iOS, Android).
High-performance LLM inference engine (MLCEngine).
OpenAI-compatible API (REST, Python, JS, iOS, Android).
ML compilation for optimized model execution.
Detailed Introduction
MLC LLM is a sophisticated machine learning compiler and a high-performance deployment engine specifically designed for large language models. Its core mission is to democratize AI by enabling developers to optimize and deploy LLMs natively across a wide array of hardware platforms, including various GPUs (AMD, NVIDIA, Apple, Intel), web browsers, and mobile devices (iOS, Android). It achieves this through MLCEngine, a unified inference engine offering an OpenAI-compatible API, ensuring consistent high performance and broad accessibility.