LibreChat

LibreChat is a modern, open-source chat client that supports multiple AI providers. By adding FinOps as a custom provider, you get access to any model configured in FinOps through a familiar chat interface, plus governance features like virtual keys and built-in observability.

Setup

1. Install LibreChat

Follow the LibreChat documentation for local setup. There are multiple installation options (Docker, npm, etc.).

2. Add FinOps as a Custom Provider

Add the following to your librechat.yaml file:

custom:
 - name: "FinOps"
 apiKey: "dummy"
 baseURL: "{AI_GATEWAY_URL}/v1"
 models:
 default: ["openai/gpt-4o"]
 fetch: true
 titleConvo: true
 titleModel: "openai/gpt-4o"
 summarize: false
 summaryModel: "openai/gpt-4o"
 forcePrompt: false
 modelDisplayLabel: "FinOps"
 iconURL: /media/architecture.png

3. Docker Networking

Choose the correct baseURL for your setup:

4. Run LibreChat

Start LibreChat. FinOps will appear as a provider with all configured models available.

Virtual Keys

When FinOps has virtual key authentication enabled, set apiKey to your virtual key:

apiKey: "bf-your-virtual-key-here"

This lets you enforce usage limits, budgets, and access control per user or team. For team deployments, create a separate virtual key for each team or environment - each key can have its own rate limits, budgets, and provider access rules configured in the FinOps dashboard.

Model Selection

LibreChat displays models from the models.default list or fetches them from FinOps when models.fetch is enabled. Use FinOps model IDs in provider/model format to access any configured provider:

models:
 default:
 - "openai/gpt-5"
 - "anthropic/claude-sonnet-4-5-20250929"
 - "gemini/gemini-2.5-pro"
 - "groq/llama-3.3-70b-versatile"
 fetch: true

• Use powerful models like openai/gpt-5 or anthropic/claude-sonnet-4-5-20250929 for complex conversations
• Use fast models like groq/llama-3.3-70b-versatile for quick responses
• Set titleModel and summaryModel to lighter models to reduce cost for metadata generation

Using Multiple Providers

FinOps routes requests to the correct provider based on the model name. Use the provider/model-name format to access any configured provider through the single /v1 endpoint:

anthropic/claude-sonnet-4-5-20250929
openai/gpt-5
gemini/gemini-2.5-pro
mistral/mistral-large-latest

Supported Providers

FinOps supports the following providers with the provider/model-name format:

openai, azure, gemini, vertex, bedrock, mistral, groq, cerebras, cohere, perplexity, xai, ollama, openrouter, huggingface, nebius, parasail, replicate, vllm, sgl

Observability

All LibreChat traffic through FinOps is logged. Monitor it at {AI_GATEWAY_URL}/logs - filter by provider, model, or search through conversation content to track usage across your team.

Next Steps

• Provider Configuration - Configure AI providers in FinOps
• Virtual Keys - Set up usage limits and access control