LiteLLM SDK
Use FinOps as a drop-in proxy for LiteLLM applications with zero code changes.
Since LiteLLM already provides multi-provider abstraction, FinOps adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup.
Endpoint: /litellm
Provider Compatibility: This integration only works for AI providers that both LiteLLM and FinOps support. If you're using a provider specific to LiteLLM that FinOps doesn't support (or vice versa), those requests will fail.
Setup
Python
from litellm import completion
# Configure client to use FinOps
response = completion(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}],
base_url="{AI_GATEWAY_URL}/litellm" # Point to FinOps
)
print(response.choices[0].message.content)Provider/Model Usage Examples
Your existing LiteLLM provider switching works unchanged through FinOps:
Python
from litellm import completion
# All your existing LiteLLM patterns work the same
base_url = "{AI_GATEWAY_URL}/litellm"
# OpenAI models
openai_response = completion(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello GPT!"}],
base_url=base_url
)
# Anthropic models
anthropic_response = completion(
model="claude-3-sonnet-20240229",
messages=[{"role": "user", "content": "Hello Claude!"}],
base_url=base_url
)
# Google models
google_response = completion(
model="gemini/gemini-1.5-flash",
messages=[{"role": "user", "content": "Hello Gemini!"}],
base_url=base_url
)
# Azure models
azure_response = completion(
model="azure/gpt-4o",
messages=[{"role": "user", "content": "Hello Azure!"}],
base_url=base_url
)Adding Custom Headers
Add FinOps-specific headers for governance and tracking:
Python
from litellm import completion
# Add custom headers for FinOps features
response = completion(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}],
base_url="{AI_GATEWAY_URL}/litellm",
extra_headers={
"x-bf-vk": "your-virtual-key", # Virtual key for governance
}
)
print(response.choices[0].message.content)Supported Features
The LiteLLM integration supports all features that are available in both the LiteLLM SDK and FinOps core functionality. Your existing LiteLLM code works seamlessly with FinOps's enterprise features. 😄
Next Steps
- Governance Features - Virtual keys and team management
- Semantic Caching - Intelligent response caching
- Configuration - Provider setup and API key management