Streaming Parity Across Six Backends: Where the OpenAI-Compatible Stream Diverges

Why Streaming Parity Matters

When you call multiple model backends (Claude, GPT, DeepSeek, Qwen, GLM, Kimi) through one OpenAI-compatible API, the HTTP interface is uniform, but the raw Server-Sent Events (SSE) stream may differ in subtle ways. These differences can break streaming clients that assume a single canonical format. This article explains where divergences occur and how to handle them.

The Common Baseline: OpenAI-Compatible SSE

Most providers emit SSE events with a data: line containing a JSON object. The typical structure includes choices with delta objects carrying incremental content, a finish_reason field when the stream ends, and sometimes a usage object in the final chunk. Clients parse these to render text progressively.

However, not all backends implement every field identically. Below are the main areas of divergence.

Delta Shape Variations

• Missing or null fields: Some backends omit role in the first delta or send content as null instead of an empty string. Others include toolcalls or functioncall only when needed.
• Extra fields: Backends may add provider-specific fields like reasoning_content or annotations that standard clients ignore.
• Chunk granularity: The size and number of delta chunks can vary, affecting perceived latency but not correctness.
• Unicode and escaping: JSON escaping of special characters can differ, but should be handled by a compliant parser.

Recommendation: Treat unknown fields as optional and never assume content is a string until you check.

Finish Reason Differences

• Standard values: stop, length, toolcalls, contentfilter are common, but not all backends emit every value.
• Custom reasons: Some backends may use functioncall or omit finishreason entirely until the final chunk.
• Timing: The finish_reason may appear in the same chunk as the last content delta or in a separate empty delta chunk.

Recommendation: Use finish_reason only to detect completion, but do not rely on its value for logic beyond basic checks.

Usage Reporting

• Final chunk usage: OpenAI includes usage in the final chunk when streamoptions: {"includeusage": true} is set. Other backends may always include it, never include it, or include it per chunk.
• Field presence: prompttokens, completiontokens, and total_tokens may be missing or zero in some streams.
• Alternative reporting: Some backends report usage only in a separate event or not at all for streaming.

Recommendation: If you need token counts, be prepared to fall back to non-streaming calls or estimate from text length.

Error and Termination Events

• Error chunks: Some backends send an error object as a data: event, while others close the connection abruptly.
• Heartbeats: Keep-alive comments (: ping) may be sent at different intervals.
• End of stream: The [DONE] sentinel is common but not universal; some backends simply close the connection.

Recommendation: Implement robust error handling and treat connection close as a potential end-of-stream signal.

Practical Testing Approach

To identify divergences for your use case:

1. Use a raw HTTP client (e.g., curl) to capture SSE streams from each backend.
2. Log all data: lines and inspect the JSON structure.
3. Compare across backends for the same prompt.
4. Note differences in delta shape, finish reasons, and usage.
5. Adjust your client to be permissive: ignore unknown fields, handle nulls, and do not assume usage is present.

Building a Resilient Streaming Client

• Parse each SSE event independently; do not accumulate state that assumes a fixed sequence.
• Use a JSON parser that tolerates missing fields.
• Treat finish_reason as optional and check for [DONE] or connection close.
• If usage is critical, consider a non-streaming fallback or a secondary API call.
• Test with all backends you plan to use, as divergences may appear only under certain conditions.

Conclusion

While the OpenAI-compatible API provides a consistent surface, the underlying SSE streams from different backends can vary in delta shape, finish reasons, and usage reporting. By understanding these divergences and writing tolerant clients, you can maintain reliable streaming across all models accessible with your single API key.