Tags: #openai-api-compatible
AI Inference Engine
WebGPU
17.8k
mlc-ai/web-llm
A high-performance, in-browser LLM inference engine with OpenAI API compatibility, leveraging WebGPU for local, private AI.
Replaces:
Details AI Inference Proxy
python
3.4k
algorithmicsuperintelligence/optillm
OptiLLM is an OpenAI API-compatible inference proxy that significantly boosts LLM accuracy and performance on reasoning tasks using 20+ state-of-the-art optimization techniques, requiring zero training.
CLI Tool / Local AI Inference Platform
llama.cpp
2.8k
janhq/cortex.cpp
A local AI API platform designed to run various AI models (vision, speech, language) on local hardware with an OpenAI-compatible API.
Replaces:
Details