DEVELOPER DOCUMENTATION

Streaming responses

Render output as it arrives with standard server-sent events.

{
  "model": "your-model-id",
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ],
  "max_tokens": 1024,
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}

Read each data: frame, append choices[0].delta.content, and stop on [DONE]. The final usage frame can contain no choices; keep reading until it arrives.

Cancellation and receipts

Cancel the request with an AbortController when the user stops generation. A stopped stream is billed what the provider billed for it, plus the routing fee (see failed and interrupted requests). Stream costs finalize after the response headers, so inspect Activity for the settled receipt.