Build Multi-Model AI Apps: Routing & Fallbacks

A production AI app should survive a model outage without waking anyone up. The pattern that gets you there is boring and reliable: one OpenAI-compatible client, a ranked list of models, routing rules that prefer cheap-but-healthy providers, and hard price caps.

What you will build in this tutorial:

• A single client that can call GPT, Claude, Gemini, DeepSeek and others by changing one string

• Automatic fallbacks — if the primary model errors or times out, the request retries on the next model in your list

• Cost controls — cheapest-first routing and max_price caps so a traffic spike cannot torch your budget

• Spend visibility — per-request cost accounting you can log and alert on

Everything below uses OpenRouter as the routing layer; the same architecture works with any OpenAI-compatible gateway, including APIMart when you need image, video or audio models in the same app.

Why multi-model beats single-model

Outages are a when, not an if

Every major provider has visible incidents. If your app hardcodes one vendor, every one of those incidents is your incident. A fallback chain turns "provider down" into a latency blip.

Models have different sweet spots

Cheap fast models handle classification and extraction; frontier models handle reasoning-heavy generation. Mixing tiers per task routinely cuts inference bills by half or more.

Pricing changes monthly

Model prices drop constantly. If switching models is a one-line change, you can chase the best price-performance every quarter without a migration project.

Step 1: one client, many models

Point the official OpenAI SDK at the gateway — no custom HTTP code needed:

from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-..."
)

resp = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.5",
    messages=[{"role": "user", "content": "Summarize this contract clause..."}],
)
print(resp.choices[0].message.content)

Switching to deepseek/deepseek-chat or google/gemini-2.5-pro is just a different model string. Keep model names in config, not code.

TypeScript version

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});

const resp = await client.chat.completions.create({
  model: "deepseek/deepseek-chat",
  messages: [{ role: "user", content: "Classify this ticket: ..." }],
});

A model registry, not scattered strings

Centralize the mapping from task → model tier once:

{
  "extract": "deepseek/deepseek-chat",
  "chat": "anthropic/claude-sonnet-4.5",
  "reason": "openai/gpt-5.2"
}

Step 2: fallbacks that fire automatically

The models array

OpenRouter retries the request across a ranked list server-side when the primary model errors, is rate-limited, or times out ^[1]:

{
  "model": "openai/gpt-5.2",
  "models": ["anthropic/claude-sonnet-4.5", "deepseek/deepseek-chat"],
  "messages": [{ "role": "user", "content": "..." }]
}

The response tells you which model actually served the request — log it.

Provider-level failover

Below model fallbacks, each model can be served by several providers. Routing preferences pin or exclude specific upstreams ^[2]:

{
  "model": "meta-llama/llama-3.3-70b-instruct",
  "provider": {
    "order": ["deepinfra", "together"],
    "allow_fallbacks": true
  }
}

Client-side last resort

Wrap the call in one retry against a different gateway (or a cached response) for the rare case the router itself is unreachable. Keep it to one retry — retry storms are self-inflicted outages.

Step 3: cost control that cannot be bypassed

Cheapest-first routing

Sort providers by price when latency is secondary — either per-request ("provider": {"sort": "price"}) or by using the :floor model suffix for batch jobs.

Hard price caps

max_price rejects any provider quoting above your ceiling (dollars per 1M tokens):

{
  "model": "openai/gpt-5.2",
  "max_price": { "prompt": 1.5, "completion": 10 },
  "messages": [{ "role": "user", "content": "..." }]
}

This is a guarantee, not a preference — requests that cannot be served under the cap fail fast instead of silently costing more.

Track cost per request

Responses include usage; multiply by the served model's rates and emit it as a metric. Alert on cost-per-task drift, not just total spend — drift is how a silent fallback to a pricier model shows up.

Beyond text: the same pattern for image, video and audio

LLM routers stop at language models. Real products also generate images, video and speech — and juggling five more vendor SDKs reintroduces the exact problem you just solved.

One gateway for all modalities

APIMart exposes 500+ models — chat plus GPT-Image-2, Sora 2, Kling, Veo, Suno — behind one OpenAI-compatible API and one balance.

Familiar integration, discounted rates

The client setup is identical to Step 1 with a different base URL, and per-model prices sit about 20% below official list — see the pricing page for exact per-model rates.

Mixing routers is fine

A common production layout: OpenRouter or direct APIs for text, APIMart for media generation — both behind the same abstraction in your code, both replaceable in config.

Production checklist

Ship with all five: config-driven model registry, server-side fallback chain, cheapest-first routing where latency allows, max_price caps on every call, and per-request cost metrics with drift alerts. That combination is what lets a two-person team run a multi-model app without a dedicated ops rotation.