OSS Alternative - Discover Top Open Source Alternatives to Popular Software

BerriAI/litellm

A unified open-source AI Gateway and Python SDK to call over 100 LLM APIs in an OpenAI-compatible format, offering features like cost tracking, load balancing, and guardrails.

Core Features

Unified API for 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, etc.).

Drop-in OpenAI compatibility for seamless provider swapping without code rewrite.

Production-ready gateway with virtual keys, spend tracking, guardrails, and load balancing.

Low latency performance (8ms P95 at 1k RPS) for efficient operations.

Available as a Python SDK for direct integration or a self-hosted/managed proxy server.

Quick Start

uv add litellm

Detailed Introduction

LiteLLM addresses the complexity of integrating and managing multiple Large Language Model (LLM) APIs by providing a single, unified interface. It allows developers to interact with over 100 LLM providers using the familiar OpenAI format, eliminating the need for provider-specific SDKs and authentication patterns. Beyond simplification, LiteLLM offers enterprise-grade features such as cost tracking, load balancing, guardrails, and an admin dashboard, making it a robust solution for deploying and scaling LLM applications efficiently and cost-effectively in production environments.