Litelm is a streamlined software library that facilitates routing of large language model (LLM) calls across various providers while translating between different message formats. It is designed as a minimal alternative to litellm, stripping away extensive features such as proxy servers, caching layers, and cost tracking to focus on essential functions. The core codebase consists of approximately 2,900 lines and depends only on openai and httpx libraries.
Users can install litelm with optional support for providers like Anthropic and AWS Bedrock. The API supports synchronous and asynchronous calls for completions, embeddings, streaming responses, and tool usage, maintaining compatibility with litellm’s interface for easy adoption.
Litelm routes requests to 19 different providers using a "provider/model-name" syntax and supports any OpenAI-compatible endpoint via a configurable API base URL. It also provides a consistent exception hierarchy to handle common errors such as rate limits, authentication failures, and context window limits.
The library supports advanced features including tool calls within prompts and streaming outputs, making it suitable for integrating with various LLM-powered applications. It has been tested extensively with a suite of 262 internal tests and 55 skipped tests, alongside live tests against 45 providers and DSPy integration checks.
Litelm is currently in alpha, with ongoing development and compatibility audits ensuring alignment with litellm’s routing and formatting standards. This lightweight approach offers developers a focused tool for managing LLM interactions without the overhead of additional infrastructure components.