Machine Learning Compiler and LLM Deployment Engine
22.3k 2026-03-27
mlc-ai/mlc-llm
A universal machine learning compiler and high-performance engine for deploying large language models efficiently across diverse hardware platforms.
Core Features
Universal LLM deployment across various platforms (AMD, NVIDIA, Apple, Intel GPUs, Web, iOS, Android).
High-performance LLM inference engine (MLCEngine).
OpenAI-compatible API for easy integration (REST, Python, JS, iOS, Android).
ML compilation for optimized model execution.
Detailed Introduction
MLC LLM is a groundbreaking project that provides a machine learning compiler and a high-performance deployment engine specifically designed for large language models. Its core mission is to democratize AI by enabling developers to optimize and deploy LLMs natively across a wide array of hardware, including various GPUs (AMD, NVIDIA, Apple, Intel), web browsers, and mobile devices (iOS, Android). By offering a unified MLCEngine with an OpenAI-compatible API, MLC LLM simplifies the process of bringing powerful AI models to diverse user platforms, ensuring efficient and accessible AI integration.