DEVELOPER DOCUMENTATION

Routing

Price competition across eligible providers, with explicit model mappings.

Read Provider verification for supplier selection, workflow checks, and the limits of model-identity verification.

Exact versions stay separate

Model aliases only merge where the gateway advertises an unambiguous mapping. A similarly named model, quantization, dated revision, or variant is not assumed equivalent.

Fresh capacity

Marketplace floors include healthy, available venue-attested sellers and direct venue quotes. Unknown trust is excluded. Market snapshots older than three minutes are excluded from current savings and eligibility.

When a paid request uses a venue-attested marketplace quote, its approved-provider allow-list stays attached through dispatch for both streaming and non-streaming calls. A quote without a selectable approved provider is rejected before reserving credits. Catalog refreshes cannot widen the admitted provider list.

How a route is selected

The gateway estimates input and requested maximum output tokens, prices the eligible primary routes, and selects the least expensive. Reference provider is the final fallback for the same model when discounted providers cannot serve it. Quota, availability, and transport failures can retry before output starts, within your balance, key allowance, feature requirements, and price guards.

Conversation continuity

Send the same routing_session identifier (8–128 letters, digits, underscores or hyphens) on each turn to enable provider affinity in Auto. The Playground does this automatically. A fresh quote from the previous provider is preferred when its estimated, fee-inclusive request cost is within 5% of the cheapest eligible route. Explicit discount controls take priority.

Affinity is best effort, expires after 15 minutes of inactivity, and may reset when a gateway restarts or a request reaches another replica. Marketplace pinning retains a provider host, not an individual seller account; prompt caching is not guaranteed. Unavailable or more expensive providers are skipped.

Observation versus execution

A market floor shows the best observed offer. The router quote is the ceiling used for route selection and reservation. A venue’s estimated-discount path alone does not guarantee final rates or exclude its internal fallback providers.

Account-wide savings protection

In Settings, enable Minimum quoted savings to require eligible quotes to meet your minimum after Router fees. It applies to paid requests across your account’s API keys, including coding agents and Fusions. Per-model overrides replace the default; a Fusion override takes priority over its selected model’s override. An override of 0% allows normal fallback for that model or Fusion.

Both input and output quotes must meet the minimum against fresh list prices for the same model. Every retry is checked. If no eligible quote qualifies, the request fails with minimum_savings_not_met. This filters quoted rates; it does not guarantee final-bill savings. Protection starts off and must be saved to take effect. Free requests are unaffected.

Require a minimum discount

Send min_discount in a Chat Completions request. For example, 90 requires both input and output rates to be at least 90% below their respective direct-list rates. This is a rate constraint, not a percentage calculated from the final token mix or rounded chart labels.

{
  "model": "your-model-id",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "max_tokens": 256,
  "min_discount": 90
}

Values are numbers from 0 to 100 with up to two decimal places, including 99.5. Zero requires prices no higher than direct list; omit the field for normal routing. Both streaming and non-streaming requests enforce the same policy, including sponsored :free requests.

The gateway uses quotes no older than three minutes and requires a nonzero direct reference for each rate. The executable quote must meet both limits, and the provider adapter must enforce those limits before execution across every internal fallback. An observed market floor alone does not qualify. Missing or unsupported guarantees return HTTP 503 with minimum_discount_not_met before reserving credits or calling a provider. Invalid values return HTTP 400.