What is an integration?

Overview

An integration is a protocol adapter that translates between FinOps's unified API and provider-specific API formats. Each integration handles request transformation, response normalization, and error mapping between the external API contract and FinOps's internal processing pipeline.

Integrations enable you to utilize FinOps's features like governance, MCP tools, load balancing, semantic caching, multi-provider support, and more, all while preserving your existing SDK-based architecture. FinOps handles all the overhead of structure conversion, requiring only a single URL change to switch from direct provider APIs to FinOps's gateway.

FinOps converts the request/response format of the provider API to the FinOps API format based on the integration used, so you don't have to.

Quick Migration

Before (Direct Provider)

import openai

client = openai.OpenAI(
 api_key="your-openai-key"
)

After (FinOps)

import openai

client = openai.OpenAI(
 base_url="{AI_GATEWAY_URL}/openai", # Point to FinOps
 api_key="dummy-key" # Keys are handled in FinOps now
)

That's it! Your application now benefits from FinOps's features with no other changes.

Supported Integrations

Provider-Prefixed Models

Use multiple providers seamlessly by prefixing model names with the provider:

OpenAI

import openai

# Single client, multiple providers
client = openai.OpenAI(
 base_url="{AI_GATEWAY_URL}/openai",
 api_key="dummy" # API keys configured in FinOps
)

# OpenAI models
response1 = client.chat.completions.create(
 model="gpt-4o-mini", # (default OpenAI since it's OpenAI's SDK)
 messages=[{"role": "user", "content": "Hello!"}]
)

Anthropic

import openai

# Anthropic models using OpenAI SDK format
response2 = client.chat.completions.create(
 model="anthropic/claude-3-sonnet-20240229",
 messages=[{"role": "user", "content": "Hello!"}]
)

Azure

import openai

# Azure models
response4 = client.chat.completions.create(
 model="azure/gpt-4o",
 messages=[{"role": "user", "content": "Hello!"}]
)

Vertex

import openai

# Google Vertex models
response3 = client.chat.completions.create(
 model="vertex/gemini-pro",
 messages=[{"role": "user", "content": "Hello!"}]
)

Ollama

import openai

# Local Ollama models
response5 = client.chat.completions.create(
 model="ollama/llama3.1:8b",
 messages=[{"role": "user", "content": "Hello!"}]
)

Direct API Usage

For custom HTTP clients or when you have existing provider-specific setup and want to use FinOps gateway without restructuring your codebase:

import requests

# Fully OpenAI compatible endpoint
response = requests.post(
 "{AI_GATEWAY_URL}/openai/v1/chat/completions",
 headers={
 "Authorization": f"Bearer {openai_key}",
 "Content-Type": "application/json"
 },
 json={
 "model": "gpt-4o-mini",
 "messages": [{"role": "user", "content": "Hello!"}]
 }
)

# Fully Anthropic compatible endpoint
response = requests.post(
 "{AI_GATEWAY_URL}/anthropic/v1/messages",
 headers={
 "Content-Type": "application/json",
 },
 json={
 "model": "claude-3-sonnet-20240229",
 "max_tokens": 1000,
 "messages": [{"role": "user", "content": "Hello!"}]
 }
)

# Fully Google GenAI compatible endpoint
response = requests.post(
 "{AI_GATEWAY_URL}/genai/v1beta/models/gemini-1.5-flash/generateContent",
 headers={
 "Content-Type": "application/json",
 },
 json={
 "contents": [
 {"parts": [{"text": "Hello!"}]}
 ],
 "generation_config": {
 "max_output_tokens": 1000,
 "temperature": 1
 }
 }
)

Listing Models

All integrations support listing available models through their respective list models endpoints (e.g., /openai/v1/models, /anthropic/v1/models). By default, list models requests return models from all configured providers in FinOps.

Filtering by Provider

You can control which provider's models to list using the x-bf-list-models-provider header:

Python

import openai

client = openai.OpenAI(
 base_url="{AI_GATEWAY_URL}/openai",
 api_key="dummy-key"
)

# List models from all providers (default behavior)
all_models = client.models.list

# List models from a specific provider only
openai_models = client.models.list(
 extra_headers={
 "x-bf-list-models-provider": "openai"
 }
)

anthropic_models = client.models.list(
 extra_headers={
 "x-bf-list-models-provider": "anthropic"
 }
)

JavaScript

import OpenAI from "openai";

const openai = new OpenAI({
 baseURL: "{AI_GATEWAY_URL}/openai",
 apiKey: "dummy-key",
});

// List models from all providers (default behavior)
const allModels = await openai.models.list;

// List models from a specific provider only
const openaiModels = await openai.models.list({
 headers: {
 "x-bf-list-models-provider": "openai",
 },
});

const anthropicModels = await openai.models.list({
 headers: {
 "x-bf-list-models-provider": "anthropic",
 },
});

cURL

# List models from all providers (default)
curl {AI_GATEWAY_URL}/openai/v1/models

# List models from specific provider
curl {AI_GATEWAY_URL}/openai/v1/models \
 -H "x-bf-list-models-provider: openai"

# Explicitly request all providers
curl {AI_GATEWAY_URL}/openai/v1/models \
 -H "x-bf-list-models-provider: all"

Header Behavior

Header Value	Behavior
Not set (default)	Lists models from all configured providers
`all`	Lists models from all configured providers
`openai`	Lists models from OpenAI provider only
`anthropic`	Lists models from Anthropic provider only
`vertex`	Lists models from Vertex AI provider only
Any valid provider	Lists models from that specific provider

Start with development - Test FinOps in dev environment
Canary deployment - Route 5% of traffic through FinOps
Feature-by-feature - Migrate specific endpoints gradually
Full migration - Switch all traffic to FinOps

Blue-Green Migration

import os
import random

# Route traffic based on feature flag
def get_base_url(provider: str) -> str:
 if os.getenv("USE_BIFROST", "false") == "true":
 return f"http://finops:8080/{provider}"
 else:
 return f"https://api.{provider}.com"

# Gradual rollout
def should_use_bifrost -> bool:
 rollout_percentage = int(os.getenv("BIFROST_ROLLOUT", "0"))
 return random.randint(1, 100) <= rollout_percentage

Feature Flag Integration

# Using feature flags for safe migration
import openai
from feature_flags import get_flag

def create_client:
 if get_flag("use_bifrost_openai"):
 base_url = "http://finops:8080/openai"
 else:
 base_url = "https://api.openai.com"

 return openai.OpenAI(
 base_url=base_url,
 api_key=os.getenv("OPENAI_API_KEY")
 )

Next Steps

HTTP Transport Overview - Main HTTP transport guide
Endpoints - Complete API reference
Configuration - Provider setup and config