How to stream responses from an OpenAI-compatible API
Learn Chat Completions streaming, SSE events, and delta fields. Use a JavaScript example to handle response chunks, token usage, and interruptions.
What changes with streaming?
A standard request waits for the complete response. With streaming, small pieces arrive incrementally, allowing your interface to display text as it is generated. This is especially useful for chat and writing tools with longer responses.
Streaming does not necessarily reduce total generation time. Its benefit is showing the first useful piece earlier. Keep a loading state until content arrives, and distinguish an established connection from a completed response.
Start a stream with the SDK
First create an OpenAI client with your WhatOTP key and a baseURL ending in /v1, as shown in the getting-started guide. Then send stream: true in your Chat Completions request. The JavaScript SDK exposes an async iterator you can consume with for await.
Not every chunk contains text. The choices array can be empty, or a delta may only carry additional information. Use optional access when reading fields and append content only when it is present.
const stream = await client.chat.completions.create({
model: "gpt-6-astra",
messages: [{ role: "user", content: "Explain HTTP streaming." }],
stream: true,
stream_options: { include_usage: true },
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content;
if (content) process.stdout.write(content);
if (chunk.usage) console.log("Usage:", chunk.usage);
}Understand SSE and delta fields
At the HTTP level, the response uses text/event-stream. Lines starting with data: carry JSON events, while [DONE] indicates completion. choices[0].delta.content is the new piece of text, not the entire response. Append pieces in order rather than replacing previous output.
When using fetch directly instead of the SDK, network chunks do not match event boundaries. One JSON event can span multiple reads, and one read can contain several events. Buffer incoming data until complete SSE events are available and use a streaming UTF-8 decoder.
Handle token usage separately
Send stream_options: { include_usage: true } to request usage information in the stream. An event containing usage might not contain response text. That is why the example checks usage and content independently.
If the provider does not report usage, the dashboard can show estimated token counts. Do not treat the number of output characters as an exact token count. For troubleshooting, correlate the X-Request-Id response header with the request history entry.
Represent interrupted responses accurately
Close the active connection when a user leaves the page or cancels generation. If the stream stops unexpectedly, do not mark existing text as a completed response. You can retain the partial answer and offer a retry option.
An error after HTTP headers have been sent may not be represented by a new HTTP status code. Track completion of the stream rather than relying only on the initial 200 response. Use distinct application states for network errors, user cancellation, and successful completion.