When an Error Arrives Mid-Stream: Reading Error Events Inside an SSE Response Instead of a Clean HTTP Failure

Why Streaming Errors Are Different

When you set stream: true in an API request, the server sends an HTTP 200 OK response immediately, before the model has generated any content. This means that any failure occurring after the headers are sent—such as an invalid request, rate limiting, or an internal server error—cannot be reported via HTTP status codes. Instead, the API embeds error information as special events within the Server-Sent Events (SSE) stream.

Understanding how to detect and interpret these error events is essential for robust client applications.

The SSE Event Structure

SSE responses consist of a sequence of events. Each event is a block of lines, typically with a data: prefix, separated by blank lines. For example:

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","choices":[{"delta":{"content":" world"}}]}

data: [DONE]

In a successful stream, you receive a series of JSON chunks followed by a [DONE] sentinel.

But when an error occurs mid-stream, the API sends an event that contains an error payload instead of a normal completion chunk. This payload often has an error field at the top level or within the JSON structure.

How to Detect Error Events

To catch errors embedded in the stream, your client must parse each event's data and check for the presence of an error key or other error indicators. Here are common patterns:

• Top-level error object: The event data is a JSON object with an error property, e.g., {"error": {"message": "Invalid model", "type": "invalidrequesterror"}}.
• HTTP status code in the payload: Some APIs include a status or code field in the error object.
• Special event names: Some SSE implementations use event: error to signal an error, with the error details in the data: line.

Always refer to the specific API documentation for the exact error format. In general, if the parsed JSON contains an error field, treat it as an error condition.

Handling the Error

Once you detect an error event, you should:

1. Stop reading the stream to avoid processing further data that may be incomplete or invalid.
2. Extract the error details (message, type, code) from the payload.
3. Log or display the error appropriately.
4. Implement retry logic if the error is transient (e.g., rate limiting).

Note that after an error event, the server may close the connection or send a [DONE] sentinel. Your client should handle both cases gracefully.

Example: Parsing SSE in JavaScript

Here's a simplified example using the fetch API and a manual SSE parser:

const response = await fetch('https://api.example.com/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer YOUR_KEY' },
  body: JSON.stringify({ model: 'claude', messages: [{ role: 'user', content: 'Hello' }], stream: true })
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop(); // keep incomplete line
  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = line.slice(6).trim();
      if (data === '[DONE]') {
        // stream complete
        return;
      }
      try {
        const parsed = JSON.parse(data);
        if (parsed.error) {
          console.error('Stream error:', parsed.error);
          // handle error, possibly retry
          return;
        }
        // process normal chunk
        console.log(parsed.choices[0].delta.content);
      } catch (e) {
        console.warn('Failed to parse SSE data:', data);
      }
    }
  }
}

This example shows the core logic: accumulate data, split by newlines, parse each data: line, and check for errors.

Best Practices

• Always validate the response: Even if the HTTP status is 200, parse the stream for errors.
• Use a robust SSE parser: Libraries exist for most languages, but many are simple enough to implement manually.
• Handle partial chunks: SSE data may arrive split across TCP packets; buffer accordingly.
• Set a timeout: If the stream stalls, abort after a reasonable period.
• Log raw events: For debugging, keep the raw SSE lines until you've confirmed success.

Conclusion

Errors in streaming responses are not communicated via HTTP status codes; they are embedded as events within the SSE stream. By understanding the event structure and implementing proper parsing, you can detect and handle these errors gracefully, ensuring your application remains robust even when failures occur mid-stream.