OpenRouter
Unified API to 400+ LLMs from 70+ providers through one OpenAI-compatible endpoint, with automatic failover and pass-through token pricing
USAGE-BASED Verified JUN 23, 2026
LiteLLM
Open-source LLM gateway that puts one OpenAI-compatible API in front of 100+ model providers, with virtual keys, budgets, spend tracking and fallbacks
LiteLLM is an open-source LLM gateway that puts one OpenAI-compatible API in front of 100+ model providers, adding virtual keys, budgets, spend tracking and automatic fallbacks. The self-hosted core is MIT-licensed and listed at $0 with no usage metering or seat caps. Enterprise adds SSO beyond 5 users, SCIM, audit logs and JWT auth on a quote-only self-hosted license with a 30-day trial. Best for platform teams giving many developers governed LLM access.
Verified JUL 20, 2026 OPEN-SOURCE Live
What is LiteLLM?
LiteLLM is an LLM gateway: a server you run in front of your own model provider accounts so that every application in your company talks to one endpoint instead of a dozen different APIs. Your code sends a request in the OpenAI format, names a model, and LiteLLM forwards it to OpenAI, Anthropic, Azure, Bedrock, Vertex AI, or any of 100+ supported providers using keys you supply. Because the request and response shapes are normalised, adding or switching a model is usually a configuration change rather than an integration rewrite. It ships in two forms — a Python SDK you import into an application, and a proxy server with an admin web UI that a platform team operates centrally.
The reason teams adopt it is governance rather than routing. LiteLLM issues virtual keys that developers use instead of raw provider keys, attaches budgets and rate limits to each key, user, team or organisation, and tracks spend per request so costs can be charged back to whoever incurred them. It retries and falls back to an equivalent model on another provider when one has an outage, and streams request logs to Langfuse, Langsmith, Arize, OpenTelemetry, S3 or GCS. The core is MIT-licensed and free to self-host with no metering or user cap, and the project sends no telemetry back to the vendor when you run it yourself. A separate Enterprise license — sold by BerriAI, a Y Combinator company, and priced only on request — adds SSO beyond five users, SCIM, JWT authentication, audit logs, key rotation and multi-region deployment.
Who is it for?
LiteLLM is infrastructure for the people who hand out LLM access, not for end users writing prompts. Its value appears when more than a handful of developers or applications need models and somebody has to answer who spent what, which vendor a prompt reached, and what happens when that vendor goes down. The free build covers all of that; the paid license is essentially about identity and audit, so the decision point for most organisations is whether they need SSO for more than five people. Teams that would rather not run Postgres and a proxy themselves should note there is no first-party managed cloud, unlike hosted alternatives such as OpenRouter.
- Platform and infrastructure engineers standing up a shared internal AI gateway so product teams get day-one access to new models without each one integrating a provider SDK.
- FinOps and engineering managers who need per-team, per-project and per-tag spend attribution and hard budget caps on inference costs.
- Application and agent developers who want provider-agnostic code, automatic failover, and the ability to swap a model by editing a config file.
- Regulated and enterprise organisations needing air-gapped or multi-region self-hosting, SSO and SCIM, and audit logs over every admin action — the features that sit behind the paid license.
- Teams already running Langfuse or local models via Ollama who want one governed entry point in front of both hosted and self-hosted backends.
How much does LiteLLM cost?
Starting price: $0 · Free tier: yes · Model: open-source
Pricing verified JUL 20, 2026
Price history tracked from June 2026
| Plan | Price | Includes |
|---|---|---|
| Open Source (self-host) | $0 | MIT-licensed gateway and Python SDK, unlimited usage and users · 100+ LLM provider integrations in the OpenAI format · Virtual keys, budgets, teams, and RPM/TPM rate limits · Spend tracking by request, tag, key, user, team, and org · Load balancing, fallbacks, and request/response logging · Prometheus metrics and OTEL/Langfuse/Langsmith/Arize logging · No data or telemetry sent to LiteLLM servers when self-hosted |
| Enterprise (self-hosted license) | Custom | Quote-only — priced on deployment size, 30-day trial license offered · SSO, SCIM, and OIDC/JWT authentication (SSO is free up to 5 users) · Audit logs with retention policies and management-operation logs · Organizations, delegated admin roles, and project-scoped budgets · Key rotation plus AWS/Azure/GCP/HashiCorp secret manager integrations · Per-key and per-team guardrails and log routing · Multi-region deployment and air-gapped install under one license · Dedicated Slack/Teams support channel 9am-9pm PT weekdays included · 24/7 response-time SLAs available for an additional fee · Procurement via AWS and Azure Marketplace |
What are LiteLLM's key features?
- OpenAI-compatible gateway (proxy) covering 100+ LLM providers, plus a Python SDK for in-process use
- Virtual keys with per-key, per-user, per-team and per-org budgets and RPM/TPM rate limits
- Automatic spend tracking and cost attribution across providers, with tag-based reporting
- Fallbacks, retries, cooldowns and load balancing across multiple deployments of the same model name
- Guardrail framework with custom guardrails and Presidio PII masking in the open-source build
- Logging callbacks to Langfuse, Langsmith, Arize Phoenix, OpenTelemetry, S3 and GCS
- Prometheus metrics and a self-hosted Admin UI for keys, teams and spend dashboards
- Pass-through endpoints, batches API support, and MCP and agent registries
- Deployment via Docker Compose, Kubernetes, Helm, or the pip package
What people use LiteLLM for
- 01 Giving hundreds of internal developers governed access to OpenAI, Azure, Bedrock, Vertex and Anthropic models through one endpoint and one key
- 02 Charging LLM spend back to the right team, project or customer using tagged virtual keys and per-key budgets
- 03 Keeping an application online during a provider outage by failing over to an equivalent model on another vendor
- 04 Swapping model providers without rewriting application code, because every provider is exposed in the OpenAI request format
- 05 Enforcing hard budgets and rate limits so a runaway agent loop cannot burn an entire month of inference spend
- 06 Routing per-team request logs into Langfuse, Langsmith, Arize, S3 or GCS for observability and compliance
Pros and cons
| Pros | Cons |
|---|---|
| The gateway core is genuine MIT — self-hosting is free, unmetered, uncapped in users, and sends no telemetry back to the vendor | Enterprise pricing is never published — the pricing section only offers a Get In Touch button, and the docs say cost depends on deployment size and must be scoped with sales |
| You keep your own provider contracts and keys, so LiteLLM takes no cut of inference spend and adds no token markup | Not fully open source: everything in the enterprise/ directory sits under a separate BerriAI Enterprise License that forbids production use without a paid subscription |
| One OpenAI-format endpoint covers 100+ providers, so switching or adding a model is usually a config change rather than a code change | SSO is free only up to 5 users; beyond that, SSO, SCIM, JWT auth and audit logs all require the paid license, which is the usual blocker for mid-size teams |
| Cost attribution, budgets and rate limits are in the free tier, which is the hard part of handing LLM access to a large org | Only the 4 most recent stable minor lines get patches, so a self-hosted deployment must be upgraded regularly or it stops receiving fixes |
| Very active project — over 54,000 GitHub stars, a roughly weekly minor release cadence, and commits landing daily | Running it in production means owning Postgres, the proxy, and upgrades — there is no first-party managed cloud to offload that to |
| Several built-in guardrail callbacks, including LlamaGuard, LLM Guard, Lakera and Aporia prompt-injection checks, are license-gated even though the guardrail framework itself is free |
What are the best LiteLLM alternatives?
Langfuse
Open-source LLM engineering platform for tracing, observability, evals, and prompt management — self-host free or run on Langfuse Cloud
OPEN-SOURCE Verified JUL 15, 2026
Ollama
Free, open-source tool to run open-weight LLMs locally via CLI, desktop, or API — with an optional paid cloud for larger models
FREEMIUM Verified JUN 23, 2026
LangChain
Open-source frameworks for building LLM agents, plus the commercial LangSmith platform for tracing, evaluation, and deployment
FREEMIUM Verified JUL 12, 2026
Together AI
AI cloud for running, fine-tuning, and deploying open-source models via serverless inference and on-demand GPU clusters
USAGE-BASED Verified JUN 24, 2026
How people make money with LiteLLM
- Offer LLM platform consulting — deploy a self-hosted LiteLLM gateway for a client, wire up virtual keys, team budgets, guardrails and provider fallbacks, then bill a fixed-scope setup engagement plus a monthly retainer for upgrades and model onboarding
- Build a multi-tenant AI product on a self-hosted gateway — attribute token spend per customer with tagged virtual keys, then rebill that usage at a margin while keeping provider costs fully auditable per account
Frequently asked questions
Is LiteLLM free?
Yes. The LiteLLM gateway and Python SDK are MIT-licensed and free to self-host, with no usage metering, no seat caps, and no telemetry sent back to the vendor. The pricing section lists Open Source at zero cost. A separate Enterprise license, quoted on request, unlocks SSO beyond five users, SCIM, audit logs and JWT authentication.
Is LiteLLM fully open source?
Not entirely. The root LICENSE file grants MIT rights to everything outside the enterprise directory. Content inside enterprise/ falls under a separate BerriAI Enterprise License that forbids production use without a paid subscription, although copying and modifying it for development and testing is permitted. The gateway itself is genuinely MIT; the enterprise layer is source-available.
How much does LiteLLM Enterprise cost?
LiteLLM does not publish an Enterprise price. The pricing section shows only a Get In Touch button, and the documentation states that cost depends on deployment size and has to be scoped with the sales team. A 30-day enterprise trial license is offered, and procurement is available through the AWS and Azure marketplaces.
What is an LLM gateway and why would a team need one?
An LLM gateway is a server that sits between your applications and model providers. Your code calls one endpoint in a single format, and the gateway forwards each request to OpenAI, Anthropic, Azure or Bedrock using your own keys. Along the way it tracks spend, enforces budgets and rate limits, logs requests, and retries elsewhere when a provider fails.
How is LiteLLM different from OpenRouter?
OpenRouter is a hosted marketplace that resells tokens — you buy credits from OpenRouter and it bills per token plus a platform fee. LiteLLM is software you run yourself, pointed at your own provider accounts, so you keep paying OpenAI, Anthropic or Azure directly and LiteLLM takes no cut of inference.
Which LiteLLM features require a paid license?
SSO for more than five users, SCIM and OIDC/JWT authentication, audit logs with retention policies, organizations and delegated admin roles, key rotation, secret manager integrations, per-team log routing, multi-region deployment, and several built-in guardrail callbacks. Virtual keys, budgets, spend tracking, fallbacks, Prometheus metrics and request logging all remain free in the open-source build.
Is LiteLLM actively maintained?
Yes. The BerriAI repository had commits landing on 20 July 2026 and published release 1.93.0 to PyPI the day before, on a roughly weekly minor-release cadence. It carries over 54,000 GitHub stars. The project supports only the four most recent stable minor lines, so older self-hosted deployments need regular upgrades to keep receiving patches.
Does LiteLLM host models itself?
No. LiteLLM does not run inference and does not sell tokens. It routes requests to whichever providers you configure with your own API keys, including self-hosted backends. That means your model bills still come from OpenAI, Azure, Bedrock or your own GPU infrastructure, and LiteLLM only handles routing, governance and logging.