GUIDES

LLM Routers Explained: Model Routing vs Provider Routing

GPU Router teamOctober 4, 202611 min read

On the evening of October 4, 2026, a request for Claude Sonnet 5 had three possible routes in our catalog, and the cheapest was quoted at $0.5040 per million input tokens, about a quarter of the $2 list price. Choosing among routes like these is one of two jobs that go by the name LLM routing. The other is choosing the model itself.

GPU Router, our product, is an LLM router of the first kind: a provider router. It's an OpenAI-compatible API that sends each request to the lowest-cost eligible route for the exact model you name, so the bill can drop without changing the model you asked for. If that's the trade you want, it's the router we'd point you to.

Below is every route our catalog priced at that moment for five models from Anthropic, OpenAI and Google, plus a request we sent through one of them and its receipt. For Claude Sonnet 5, the cheapest route was 74.8% below Anthropic's list price. Quotes move, so read the numbers as a snapshot.

What is an LLM router?

An LLM router is a layer between your application and language models that decides, request by request, where each prompt goes. Model routers choose which model answers, trading some quality risk for a lower bill. Provider routers keep the model you named and choose which provider serves it, on price, uptime or speed.

The term has a third meaning in self-hosted inference. There, a router such as llm-d's spreads requests across the servers running one model, weighing KV-cache locality and load. That's a cluster-operations problem. This guide covers the routers you meet when you buy model access through an API.

What are the main types of LLM routers?

You'll run into three: model routers that move easy prompts to a cheaper model, provider routers that buy the same model from the cheapest or healthiest provider, and self-hosted proxies that apply your own rules across your own keys. Plenty of products combine two of them, so check what each one decides.

Model routers: easy prompts go to a cheaper model

One research example is LMSYS's RouteLLM. Its 2024 study trained routers to choose between a strong, expensive model and a weaker, cheap one. Against sending everything to GPT-4, it reported cost reductions "of over 85% on MT Bench, 45% on MMLU, and 35% on GSM8K" while keeping 95% of GPT-4's performance.

Hosted versions exist too. Amazon Bedrock's intelligent prompt routing predicts the response quality of each model in one family for each request and sends it to the best one; AWS notes it's only optimized for English prompts. OpenRouter's Auto Router (openrouter/auto) sorts each prompt into one of about 30 task types, then picks from the models OpenRouter users spend the most on for that task over the trailing seven days.

Every model router changes which model writes the answer. That's the point and the risk: the saving depends on how many of your prompts the cheaper model handles well, as the spread in RouteLLM's results shows. Test one on your own prompts before you trust it.

Provider routers: same model, cheaper or healthier route

Many models are sold by more than one provider, at different prices. A provider router picks among them. OpenRouter's docs say its default is to "load balance requests across providers, prioritizing price": it favors providers without significant outages in the last 30 seconds, then picks among the cheapest, weighted by the inverse square of price. Setting sort: "price" makes it strictly cheapest-first.

GPU Router sends each request to the least expensive eligible route for the exact model ID you send. If the discounted routes fail, the Reference provider at list price is the last fallback for the same model. Failures before the response starts can retry, but a response that's already started isn't restarted. Our routing docs explain the order.

Self-hosted proxies: your rules, your keys

LiteLLM's Router is one open-source option. It spreads calls across your own deployments and providers with cooldowns, fallbacks, timeouts and retries. Strategies include simple-shuffle (the default), latency-based-routing, least-busy and cost-based-routing. You pay each provider your own rates, and our LiteLLM vs OpenRouter comparison shows how to run it in front of a hosted router.

How much can an LLM router save?

A model router saves by moving prompts to a cheaper model, so the saving grows with the share of prompts that model handles well. A provider router saves only when the same model is sold for less somewhere. In our October 4 snapshot, the cheapest eligible route for each of the five models below was 56.9% to 74.8% below list.

Live prices from gpurouter.io/catalog/models, retrieved October 4, 2026 at 5:24 p.m. PDT (00:24:39 UTC on October 5). USD per 1M tokens, input / output; quotes include our routing fee. Prices change.

"Below list" is (1 − our price ÷ list price) × 100 on 1M input plus 1M output tokens. The Claude list prices match Anthropic's pricing page; the other two are the direct list prices in our catalog.

Routes for one model can sit far apart. One of GPT-6.1 Sol's routes quoted $2.363 per million input tokens, above the $2 list price, while another quoted $0.5460. A router that picks by price skips the expensive one.

The spread also moves. Claude Haiku 4.5's best quote was 26.4% below list in our Claude API pricing snapshot on September 30, and 72.7% here.

Across the whole catalog, 208 models had a priced route in this snapshot. For 91 of them, two or more eligible routes were competing for each request before the fallback; the other 117 had one. Among the 179 whose cheapest route publishes a direct list price, the median quote was 79.6% below list, and 31 were less than 50% below.

Why are some routes cheaper at all? Teams reserve capacity and prepay for usage they don't always need, and suppliers can sell that spare capacity for less. How much of it exists varies by model and changes over time.

A month of traffic, routed three ways

Model routing and provider routing don't compete; they stack. Take a workload that sends 40M input tokens to Claude Opus 5.5 each month and gets 8M output tokens back. Suppose a model router decides 60% of that traffic can go to Claude Sonnet 5 instead. The 60% is our assumption, not a measurement.

The routed rows use the October 4 quotes above. In this snapshot, provider routing saved more than model routing and kept Opus 5.5 on every request. Routed Opus 5.5, at $1.05 / $5.25, even cost less per token than Sonnet 5 at its $2 / $10 list price. If you run a model router, give it the prices you actually pay, not list prices.

A quote isn't a bill, though. Your final charge depends on the route that serves each request and the tokens you actually use.

What is the difference between an LLM router and an AI gateway?

A router decides where a request goes. A gateway governs the traffic around it: keys, logging, rate limits, caching and spend controls. Cloudflare, for one, pitches its AI Gateway on visibility and control: analytics, logging, caching, rate limiting, retries and model fallback. Many products do both, so compare features rather than labels.

GPU Router is mostly a router, with a couple of controls you'd also find in a gateway. Optional redaction replaces or rejects detected emails, formatted phone numbers, payment cards and other listed patterns before dispatch; it's pattern matching, not anonymization. Our receipts store model, provider, tokens, cost, outcome and timing, not ordinary prompt and response bodies. For a side-by-side of gateways such as LiteLLM, Portkey and Cloudflare, see our OpenRouter alternatives comparison.

How do you use GPU Router as your LLM router?

Point a client that speaks OpenAI Chat Completions at https://gpurouter.io/v1, authenticate with your GPU Router key, and send the exact model ID you want. We price that model's eligible routes and send the request to the cheapest one. The quickstart walks through the same setup, and our AI SDK guide shows it with the Vercel AI SDK's OpenAI-compatible provider.

We ran this script a minute after the price snapshot (5:25 p.m. PDT):

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://gpurouter.io/v1",
    api_key=os.environ["GPU_ROUTER_API_KEY"],
)

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",  # exact model ID: we pick the route, not the model
    messages=[
        {"role": "user", "content": "In one sentence: how is model routing different from provider routing?"}
    ],
    max_tokens=64,
)

print(response.choices[0].message.content)
print("input tokens:", response.usage.prompt_tokens, "output tokens:", response.usage.completion_tokens)

It printed "Model routing selects which AI model (e.g., GPT-4, Claude, Llama) is best suited to handle a given request…" until the 64-token cap cut it off, then input tokens: 24 output tokens: 64. Raise max_tokens for complete replies. The receipt for that request charged $0.000140 with our routing fee included. At Anthropic's list price, the same 24 input and 64 output tokens cost $0.000688.

The quote we pulled a minute earlier implied $0.000168 for those tokens, so the bill came in under the quote. Activity in your account records each request's timings, and the latency docs cover receipts, including the GET /v1/receipts/{reservationId} endpoint.

Before you route real traffic, a few details:

• Fees. Catalog quotes already include our routing fee, and savings are capped at 95%.
• A privacy filter. Add extra_body={"zdr": True} to every Python call (top-level "zdr": true in JSON) to use only zero-data-retention routes. We ran that variant too, and it cost $0.000527 because ZDR routes have their own quotes: $1.575 / $7.875 for Sonnet 5 in our snapshot. If no ZDR route can serve a request, you get HTTP 503 privacy_route_unavailable, never a silent fallback. The ZDR docs have the details.
• A savings floor. Turn on minimum quoted savings in Settings, and paid requests stop with HTTP 503 minimum_savings_not_met when no eligible quote meets your minimum after fees. It's off by default, and because it filters quotes, it can't guarantee the final bill.
• Model routing, if you want it. A Fusion saves a default model plus task specialists under one Fusion model ID, then routes each request by task using those saved preferences.
• Limits. We don't serve the OpenAI Responses API or the Anthropic Messages API, so Codex and Claude Code won't work by changing their base URL. For Cursor, Zed, Continue, Cline, OpenCode and Aider, use our coding-agent guides.

When an LLM router won't help

A provider router has nothing to work with when no discounted route can serve a model. On GPU Router, the last fallback is then the Reference provider at list price plus our routing fee, about what you'd pay direct. Check the model's page before you count on a discount.

Model routers can't make a small model good at a hard prompt; they can only guess which prompts are easy. And a hosted router adds a network hop between you and the model.

Try provider routing on your own traffic

Open any model in our model catalog to see live route prices, or put two side by side on the comparison page. When you're ready, create an API key. New accounts get $1 in free prepaid credits, which would have covered about 7,000 requests like ours at the $0.000140 it cost.