LiteLLM
Open-source proxy that gives 100+ LLM providers one OpenAI-compatible API.
Visit LiteLLM Free open source; Enterprise support plans
Quick verdict
- Best for
- Engineering teams governing LLM spend across apps
- Pricing
- Free open source; Enterprise support plans
- Not ideal if
- You run and upgrade it; releases are frequent
What is LiteLLM?
LiteLLM is an open-source Python library and proxy server that lets you call OpenAI, Anthropic, Google, Bedrock, Azure, and dozens more through one OpenAI-format interface, with spend tracking, budgets, rate limits, fallbacks, and virtual keys for teams.
Key features
- One API format across 100+ providers
- Virtual keys with per-team budgets and rate limits
- Fallbacks, retries, and load balancing across deployments
- Spend logging and cost tracking
- Self-hosted proxy with an admin UI
Pros
- Self-hosted — your keys and logs stay with you
- Budgets and virtual keys solve real team governance problems
- Huge provider coverage
Cons
- You run and upgrade it; releases are frequent
- Provider quirks leak through the abstraction sometimes
- Enterprise features are paid
Best for
Engineering teams governing LLM spend across appsMulti-provider architecturesCompanies that cannot route through a third-party gateway
Read more
Related comparisons
Alternatives to LiteLLM
OpenRouter
4.7
One API for 300+ AI models — switch providers without rewriting code.
FreemiumFree credits, then pay-per-token
Released May 2023
Portkey
AI gateway with guardrails, observability, and prompt management for production apps.
FreemiumFree developer tier; Pro from $49/mo
Released August 2023
Helicone
Open-source LLM observability — one-line proxy, full request logs, cost tracking.
FreemiumFree 100k requests/mo, Pro $25/mo
Released February 2023