Reading Your First SSE Stream From the Aggregator: data:, [DONE] and the Chunks Between

What SSE Is and Why It Matters

Server-Sent Events (SSE) is the transport behind streaming responses from the aggregator's API. Instead of waiting for a complete JSON response, you get a series of text frames pushed over a single HTTP connection. Each frame starts with data: followed by a JSON payload. This lets you display tokens as they arrive, improving perceived latency.

The Raw Stream Format

When you enable streaming, the response body is a sequence of lines. A typical frame looks like:

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","created":1678901234,"model":"claude-3-sonnet","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

Key points:

• Lines begin with data: (note the space).
• The JSON payload is a chat.completion.chunk object.
• A blank line after each frame signals the end of that event.
• The stream ends with data: [DONE]—a literal string, not JSON.

Anatomy of a Chunk

Each chunk contains a choices array. Usually one element, but could be more if you requested multiple completions. Inside each choice:

• delta: an object holding the incremental content. For text models, it often has a content field with a string fragment. The first chunk may include a role field (e.g., "assistant").
• finishreason: null while the model is still generating. When generation stops, it becomes a string like "stop", "length", or "contentfilter".
• index: the position in the choices array.

Other fields like id, model, and created are consistent across chunks.

Reading the Stream Line by Line

To consume the stream, read the response body as text and split by newlines. For each line:

1. Skip empty lines (they separate events).
2. If the line starts with data: , extract the substring after that prefix.
3. If the extracted string is [DONE], stop—the stream is complete.
4. Otherwise, parse the JSON and process the delta.

Example pseudo-code:

for line in response.iter_lines():
    if not line:
        continue
    if line.startswith(b'data: '):
        payload = line[6:]
        if payload == b'[DONE]':
            break
        chunk = json.loads(payload)
        delta = chunk['choices'][0]['delta']
        if 'content' in delta:
            print(delta['content'], end='')

Knowing When the Stream Is Finished

There are two signals:

• The explicit data: [DONE] sentinel. When you see it, the stream is over and the connection may close.
• The finish_reason field in a chunk. When it becomes non-null, the model has stopped generating. However, you should still wait for [DONE] or the connection to close, as there might be additional metadata chunks.

If neither appears and the connection remains open without new data for a while, the stream may be stalled. Implement a timeout to avoid hanging indefinitely.

Handling Edge Cases

• Empty deltas: Some chunks may have an empty delta object (e.g., when only the role is sent). Check for the presence of content before appending.
• Non-content fields: In addition to content, deltas may include role, toolcalls, or functioncall. Handle them according to your needs.
• Errors: If the server encounters an error mid-stream, it may send a data: frame with an error object instead of a normal chunk. Always validate the JSON structure.
• Multiple choices: If you requested n > 1, each chunk's choices array may have multiple entries. Track deltas per index.

Practical Tips

• Use a streaming HTTP client that supports incremental reading (e.g., requests with stream=True, or httpx).
• Buffer partial lines if needed; SSE frames are newline-delimited.
• Log raw frames during development to understand the provider's chunking behavior.
• Remember that the aggregator proxies requests to various models (Claude, GPT, DeepSeek, etc.), so chunk formats are standardized as OpenAI-compatible, but minor variations may exist.

With these basics, you can reliably decode any SSE stream from the aggregator and build responsive applications.