TUTORIALS

AI SDK OpenAI-Compatible Provider Setup (Step-by-Step)

GPU Router teamOctober 7, 202614 min read

The AI SDK's OpenAI-compatible provider, @ai-sdk/openai-compatible, connects the Vercel AI SDK to any API that speaks OpenAI's Chat Completions format. You give it a name, a base URL and a key. After that, generateText, streamText and tool calls work as they do with a first-party provider, and structured output works once you set one flag.

GPU Router is that kind of endpoint, and it's what we recommend if you want one AI SDK provider for many models. One key and one base URL, https://gpurouter.io/v1, reach Claude, GPT, Gemini and the other models in our catalog that have a live route. We send each request to the least expensive eligible route for the exact model you name, which is provider routing rather than model swapping.

We ran every snippet in this guide against our live API on October 7, 2026, with ai 7.0.131 and @ai-sdk/openai-compatible 3.0.65, plus @ai-sdk/openai 4.0.87 for the first example. The eight requests billed $0.000712 in total. At the models' direct list prices they'd have cost $0.004786. Two mistakes are easy to make, and we reproduce both below so you can see what they look like.

What is the AI SDK OpenAI-compatible provider?

@ai-sdk/openai-compatible is the AI SDK's generic provider for APIs that follow OpenAI's Chat Completions format. You create an instance with createOpenAICompatible({ name, baseURL, apiKey }) and pass it model IDs. It handles text generation, streaming and tool calling, and it sends JSON schemas once you enable supportsStructuredOutputs.

You write that instance once, in under ten lines. The AI SDK docs list every option it accepts.

Should you use @ai-sdk/openai or @ai-sdk/openai-compatible?

Use @ai-sdk/openai-compatible for any endpoint that only implements Chat Completions. Since AI SDK 5, the OpenAI provider's default factory, openai('model-id'), calls the Responses API, and endpoints that only implement Chat Completions reject it. If you have to keep @ai-sdk/openai, create every model with openai.chat('model-id') instead.

That's the first mistake. We don't offer the Responses API, so here's how it fails against our endpoint. To try it, run Step 1's install below plus npm install @ai-sdk/openai, then save this as responses-trap.ts:

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const openai = createOpenAI({
  baseURL: "https://gpurouter.io/v1",
  apiKey: process.env.GPU_ROUTER_API_KEY,
});

try {
  // openai(modelId) uses the Responses API: POST /v1/responses
  await generateText({
    model: openai("anthropic/claude-sonnet-5"),
    prompt: "Hi",
    maxOutputTokens: 16,
  });
} catch (error) {
  console.log("openai():", (error as Error).message);
}

// openai.chat(modelId) uses Chat Completions: POST /v1/chat/completions
const { text } = await generateText({
  model: openai.chat("anthropic/claude-sonnet-5"),
  prompt: "Reply with the word ok.",
  maxOutputTokens: 16,
});
console.log("openai.chat():", text);

It printed:

openai(): Not Found
openai.chat(): Ok.

The default factory sent POST /v1/responses and got a 404. The chat model sent POST /v1/chat/completions and got an answer. Watch for this in projects that set OPENAI_BASE_URL: the OpenAI provider uses that variable as its default base URL, so pointing it at us while keeping openai('...') would produce the same 404.

Set up the AI SDK with GPU Router in four steps

The rest of this guide uses @ai-sdk/openai-compatible. Every file below ran unmodified on Node 24.8.0 and passed a strict TypeScript check. The route handler ran through a direct call rather than a Next.js server.

Step 1: Install the packages

npm install ai @ai-sdk/openai-compatible zod
npm install -D tsx

The snippets use top-level await, so the project has to run as an ES module. Add "type": "module" to package.json, or run npm pkg set type=module.

Step 2: Create an API key

Create a key on the API keys page. New accounts automatically get $1 in free prepaid credits with no payment required, which covers this whole guide many times over. Requests use Bearer authentication (authentication docs), and the provider adds that header from apiKey.

export GPU_ROUTER_API_KEY="your-key"

Step 3: Create the provider

Save this as gpurouter.ts:

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

export const gpurouter = createOpenAICompatible({
  name: "gpurouter",
  baseURL: "https://gpurouter.io/v1",
  apiKey: process.env.GPU_ROUTER_API_KEY,
  includeUsage: true, // token usage on streamed responses
  supportsStructuredOutputs: true, // send JSON schemas as response_format json_schema
});

What each option does:

• name labels the provider, and its camelCase form is the providerOptions key for custom fields.
• baseURL is the line that points the provider at us. The provider appends /chat/completions.
• includeUsage adds stream_options: { include_usage: true } to streaming requests, matching the request in our streaming docs.
• supportsStructuredOutputs makes Output.object() send your real schema. Skip it and you'll hit the second mistake.

Step 4: Generate text

Save this as basic.ts and run npx tsx basic.ts:

import { generateText } from "ai";
import { gpurouter } from "./gpurouter";

const { text, usage } = await generateText({
  model: gpurouter("anthropic/claude-sonnet-5"),
  prompt: 'What makes an API "OpenAI-compatible"? One sentence, under 20 words.',
  maxOutputTokens: 64,
});

console.log(text);
console.log(usage.inputTokens, "input tokens,", usage.outputTokens, "output tokens");

Our run printed:

It exposes the same request/response format (endpoints, JSON schema, auth style) as OpenAI's API.
34 input tokens, 37 output tokens

Model strings are exact catalog IDs such as anthropic/claude-sonnet-5. Every model page, Claude Sonnet 5's included, has API examples that use its ID, plus a playground. We capped output at 64 tokens to keep test runs cheap, so raise maxOutputTokens for real work.

How do you stream responses with token usage?

Call streamText, write out result.textStream as it arrives, then await result.usage once the stream ends. With includeUsage: true on the provider, each streaming request sends stream_options: { include_usage: true }, the setting our streaming docs use. In our run, result.usage then held the input and output token counts for the whole response.

Save this as stream.ts:

import { streamText } from "ai";
import { gpurouter } from "./gpurouter";

const result = streamText({
  model: gpurouter("anthropic/claude-haiku-4.5"),
  prompt: "Write a two-line poem about rate limits. No title.",
  maxOutputTokens: 64,
});

for await (const delta of result.textStream) process.stdout.write(delta);

const usage = await result.usage;
console.log("\n", usage.inputTokens, "input tokens,", usage.outputTokens, "output tokens");

Claude Haiku 4.5 streamed back:

The tokens flow like rivers wide, till walls of limits turn the tide,
Each query costs a precious breath, before we hit the rate-limit death.
 20 input tokens, 35 output tokens

For the dollar figure, each request gets a receipt with its model, tokens, cost and outcome, and Activity adds time to first token and completion (latency docs).

Call tools and get structured output

Both map onto standard Chat Completions fields: the SDK converts your tools to tools and your schema to response_format. On our side, function tools and structured JSON work on routes that advertise those features.

Tool calling

Save this as tools.ts:

import { generateText, isStepCount, tool } from "ai";
import { z } from "zod";
import { gpurouter } from "./gpurouter";

const { text, steps } = await generateText({
  model: gpurouter("anthropic/claude-sonnet-5"),
  tools: {
    getOrderStatus: tool({
      description: "Look up the shipping status of an order",
      inputSchema: z.object({ orderId: z.string() }),
      execute: async ({ orderId }) => ({ orderId, status: "shipped", eta: "2 days" }),
    }),
  },
  stopWhen: isStepCount(2),
  prompt: "Where is order A-1042? Answer in one short sentence.",
  maxOutputTokens: 64,
});

console.log(steps.map((step) => step.toolCalls.map((call) => call.toolName)));
console.log(text);

It printed:

[ [ 'getOrderStatus' ], [] ]
Order A-1042 has shipped and is expected to arrive in 2 days.

isStepCount(2) lets the SDK run the tool, send its result back and stop after the model's answer. That's two Chat Completions requests and two receipts. The first one carried 493 input tokens for a one-line prompt. Tools aren't free: Anthropic's pricing page lists a 354-token tool-use system prompt for Sonnet 5, on top of your tool schemas.

Structured output

This is the second mistake. Save this as structured.ts:

import { generateText, Output } from "ai";
import { z } from "zod";
import { gpurouter } from "./gpurouter";

const { output } = await generateText({
  model: gpurouter("openai/gpt-6.1-sol"),
  output: Output.object({
    schema: z.object({
      provider: z.string(),
      baseURL: z.string(),
      compatible: z.boolean(),
    }),
  }),
  prompt: 'Extract: "We moved our app to https://gpurouter.io/v1 and nothing else changed."',
  maxOutputTokens: 64,
});

console.log(output);

GPT-6.1 Sol returned an object that passed the schema:

{
  provider: 'GPURouter',
  baseURL: 'https://gpurouter.io/v1',
  compatible: true
}

Leave out supportsStructuredOutputs: true and the provider doesn't put your schema in the request. It falls back to response_format: { type: 'json_object' } and attaches a warning that reads "JSON response format schema is only supported with structuredOutputs". Our fetch logger caught our first attempt going out exactly like that. With the flag on, the request carried a json_schema block with strict: true, and the reply matched the Zod schema.

How do you pass custom fields like zdr: true?

Put them under providerOptions, keyed by your provider's name in camelCase. The OpenAI-compatible provider copies any key it doesn't recognize to the top level of the request body. With a provider named gpurouter, providerOptions: { gpurouter: { zdr: true } } sends "zdr": true, which tells us to use only zero-data-retention routes.

Save this as zdr.ts:

import { generateText } from "ai";
import { gpurouter } from "./gpurouter";

const { text } = await generateText({
  model: gpurouter("anthropic/claude-sonnet-5"),
  prompt: 'Summarize this support ticket in 10 words: "Refund not received after 14 days."',
  maxOutputTokens: 64,
  providerOptions: {
    gpurouter: { zdr: true }, // copied to the top level of the request body
  },
});

console.log(text);

It printed "Customer still awaiting refund 14 days post-request." A fetch logger we added for the test showed "zdr": true at the top level of the JSON body. Name the provider gpu-router and the key becomes gpuRouter.

Four ZDR rules matter in production (ZDR docs):

• Send it on every request that needs it. In AI SDK terms, that's every call, or a small wrapper that adds the providerOptions entry for you.
• If no ZDR route can serve the request, you get HTTP 503 privacy_route_unavailable. We never quietly fall back to standard handling.
• ZDR prices can differ from standard ones. This request billed $0.000253, close to Sonnet 5's list price. basic.ts, a slightly larger standard Sonnet 5 call (34 / 37 tokens), billed $0.000045.
• The serving provider's ZDR policy governs retention, and usage metadata (tokens, cost, timing, outcome) remains.

Our ZDR overview explains which requests are eligible.

Use the provider in a Next.js route handler

Move gpurouter.ts to lib/gpurouter.ts, then add app/api/chat/route.ts. The @/ import assumes a "@/*": ["./*"] path alias in tsconfig.json:

import {
  convertToModelMessages,
  createUIMessageStreamResponse,
  streamText,
  toUIMessageStream,
  type UIMessage,
} from "ai";
import { gpurouter } from "@/lib/gpurouter";

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: gpurouter("anthropic/claude-haiku-4.5"),
    messages: await convertToModelMessages(messages),
    maxOutputTokens: 64,
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

We called the handler directly with a one-message Request, without starting a Next.js server. It returned HTTP 200 and a text/event-stream of UI message chunks, the format the AI SDK's useChat hook from @ai-sdk/react reads. Older examples end with result.toUIMessageStreamResponse(), which AI SDK 7 deprecates in favor of the two helpers above.

What do these models cost through GPU Router?

At 22:51 UTC on October 7, 2026, our catalog quoted Claude Sonnet 5, Claude Haiku 4.5, GPT-6.1 Sol and Gemini 3.8 Flash at 86.7% to 98.0% below their direct list prices for standard requests. Those quotes describe that moment, not fixed prices, and they move with supply. ZDR quotes are separate and were higher for all four.

Prices per 1M tokens (input / output) from our live catalog, retrieved 2026-10-07 22:51 UTC. Quotes include our routing fee.

"Below list" is (1 − our price ÷ list price) × 100 on 1M input plus 1M output tokens. List prices come from OpenAI, Google and Anthropic's pricing pages. Google lists $0.75 / $3.75 for Gemini 3.8 Flash through December 31, 2026 and $1.50 / $7.50 starting January 1, 2027.

The quotes really do move. GPT-6.1 Sol's input quote was $0.0408 per million tokens at 22:39 UTC and $0.2652 about 12 minutes later. Our quotes include our routing fee, and savings are capped at 95% of list. When no discounted route can serve a model, the Reference provider at list price is the last fallback (routing docs), and you pay list price plus our routing fee.

Here's what the snippets in this guide actually cost, from each request's receipt:

Billed amounts include the routing fee, and the list column prices the same tokens at each vendor's list rate. Receipts record what each request actually cost, so they won't match a snapshot exactly. Our standard Sonnet 5 calls at 22:47 UTC billed at about 10% of list, while the catalog showed 8.2% of list at 22:39 and again at 22:51.

To size your own workload, use the cost calculator on any model page, or line models up in compare. Our Claude API pricing guide goes deeper on Claude pricing.

Troubleshooting the AI SDK with an OpenAI-compatible API

Normal output is capped at 65,536 tokens per request, and the limit can be lower because of your credits or upstream limits (errors docs). If you're still choosing an endpoint, our OpenRouter alternatives comparison lines up eight options, including self-hosted LiteLLM and Vercel AI Gateway, and our LiteLLM vs OpenRouter comparison covers the self-hosted route in depth.

Next step

Pick model IDs and check live prices in the model catalog, then create an API key. New accounts get $1 in free credits. Point baseURL at https://gpurouter.io/v1, and your generateText, streamText and tool code stays the same. If you also want to point an editor at us, we publish setup guides for Cursor, Zed, Continue, Cline, OpenCode and Aider in our coding-agent docs.