Reading the Raw SSE Frames: What Each `data:` Line Actually Contains

When you stream a response from the aggregator, the server sends a sequence of Server-Sent Events (SSE). Each event is a line starting with data: followed by a JSON payload. Understanding the raw bytes helps you build robust clients and debug issues.

The SSE Stream Format

SSE is a simple text protocol. The server sends lines like:

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"claude-3-5-sonnet","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"claude-3-5-sonnet","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1234567890,"model":"claude-3-5-sonnet","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Each data: line contains a JSON object (except the final [DONE]). The lines are separated by double newlines (\n\n).

Event Types and Their Meaning

The stream consists of chunks. Each chunk has an object field set to "chat.completion.chunk". The choices array contains one or more choice objects, each with an index, a delta, and a finish_reason.

• First chunk: Typically contains the role in the delta (e.g., "assistant"). No content yet.
• Subsequent chunks: Contain content in the delta, representing incremental text.
• Final chunk: Has an empty delta and a finish_reason (e.g., "stop", "length").
• Terminator: The literal string [DONE] as the data payload, signaling the end of the stream.

Some models may also send a system_fingerprint field, but this is not guaranteed.

Delta Fields: What to Expect

Within each delta object, you may see:

• role: Only in the first chunk, usually "assistant".
• content: The incremental text. Concatenate these to get the full response.
• tool_calls: If the model invokes a tool, you'll get partial tool call information, often with an index, id, type, and function with name and arguments fragments.
• functioncall: Deprecated in favor of toolcalls, but may appear for older models.

Handle each field as it arrives. For tool_calls, accumulate the arguments string across deltas.

Terminators and Stream End

The stream ends when you receive a chunk with finish_reason set to a non-null value, followed by a data: [DONE] line. Your client should stop reading after [DONE].

If the connection drops unexpectedly, you may not get [DONE]. Implement error handling and consider retrying with a new request.

What Your Client Should Do

• Parse each data: line as JSON, except for [DONE].
• Append delta.content to your output buffer.
• Handle delta.tool_calls by accumulating fragments.
• Watch for finish_reason to know when the model is done.
• Close the connection upon [DONE].
• Implement timeouts and error recovery.

Example: Handling a Tool Call

data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"call_abc","type":"function","function":{"name":"get_weather","arguments":""}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"location\""}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":":\"Paris\"}"}}]}}]}

You would concatenate the arguments fragments to get {"location":"Paris"}.

Conclusion

Understanding the raw SSE frames lets you build reliable streaming clients. Always check for the [DONE] terminator, handle partial deltas, and be prepared for tool calls. The aggregator follows the OpenAI-compatible chunk format, so existing SSE parsers can be adapted easily.