algorithmicsuperintelligence/optillm

OptiLLM is an innovative optimizing inference proxy designed to dramatically enhance the accuracy and performance of Large Language Models (LLMs) on complex reasoning tasks. By implementing over 20 state-of-the-art techniques, including advanced methods like MCTS and planning, it achieves 2-10x improvements without any model training or fine-tuning. It acts as a drop-in replacement for OpenAI-compatible API endpoints, making it easy to integrate into existing workflows. Supporting multiple LLM providers, OptiLLM allows users to leverage frontier model capabilities from less powerful models, effectively bridging performance gaps through intelligent inference-time computation.