Reconnecting a Broken Stream Mid-Response on One Key: Which Backends Let You Resume From the Cut

Why mid-stream reconnection matters

Streaming responses are long-lived HTTP connections. Network glitches, client timeouts, or server-side idle limits can cut the stream before completion. Without resume support, your only option is to regenerate the response from scratch, wasting tokens and time. With resume support, you can continue from the last received chunk.

The challenge: you're calling multiple backends (Claude, GPT, DeepSeek, Qwen, GLM, Kimi) through one API key. Each backend handles streaming differently. Some offer explicit resume mechanisms; others treat a broken stream as a failed request.

How streaming APIs typically handle interruptions

Most LLM streaming APIs use server-sent events (SSE). When the connection drops, the server may:

• Forget the request and discard any partial output.
• Keep the request alive for a short grace period, allowing a reconnect with a continuation token.
• Support explicit continuation via a request parameter that references a previous response ID or a sequence number.

Resume capability depends on the backend's implementation, not on the aggregator layer. Your single key routes to the same backend APIs, so the resume semantics are those of the underlying provider.

Which backends support resuming from a partial response

Based on public API documentation and observed behavior:

• OpenAI (GPT models): Supports resuming via the previousresponseid parameter in the Responses API. When a stream breaks, you can send a new request referencing the last response ID and the API will continue from where it left off. For chat completions, there is no native resume; you must resend the conversation with the partial assistant message and ask the model to continue.
• Anthropic (Claude): Does not offer a native resume token for streaming. A broken stream requires reissuing the request. You can include the partial assistant text in the conversation and prompt the model to continue, but this is a new generation, not a true resume.
• DeepSeek: No documented resume mechanism for streaming. Treat a broken stream as a failed request and regenerate.
• Qwen (Alibaba): No documented resume mechanism. Regenerate on failure.
• GLM (Zhipu): No documented resume mechanism. Regenerate on failure.
• Kimi (Moonshot): No documented resume mechanism. Regenerate on failure.

In practice, only OpenAI's Responses API currently provides first-class resume from a partial response. For all other backends, you must implement a workaround: capture the partial output and send a follow-up request that includes it, asking the model to continue. This is not a true resume—it's a new generation with context—but it avoids discarding prior tokens.

How to detect whether a backend can resume

Check the API documentation for your target model. Look for:

• Parameters like previousresponseid, continuationtoken, or resumefrom.
• Endpoints that explicitly mention "resume" or "continue".
• Error codes that indicate a resumable stream (e.g., stream_interrupted with a resume token).

If none of these exist, assume no native resume. You can also test empirically: start a stream, kill the connection mid-response, then attempt to reconnect with any received token. If the backend returns an error or starts from the beginning, resume is unsupported.

Implementing resume with one key

When using an aggregator that routes to multiple backends, you need a unified strategy:

1. Tag each request with the backend model name so you know which resume semantics apply.
2. For OpenAI Responses API models, store the responseid from the initial stream. On failure, send a new request with previousresponse_id set to that ID and input containing only the new user message. The API will continue the response.
3. For all other models, capture the partial assistant text. On failure, send a new request with the full conversation including the partial text, and append a system or user instruction like "Continue exactly where you left off." This generates a continuation but is not a true resume.
4. Set client-side timeouts slightly longer than the backend's idle timeout to avoid premature disconnects.
5. Use exponential backoff for reconnection attempts, but cap retries to avoid infinite loops.

Detecting a broken stream

A stream is broken when:

• The HTTP connection closes before the [DONE] sentinel (or equivalent) is received.
• No data arrives for a period exceeding your read timeout.
• The server returns an error event mid-stream.

Instrument your client to log the last received chunk index and any finish reason. This helps distinguish a clean end from an interruption.

Cost implications

True resume avoids re-billing for tokens already generated. Workaround continuations re-send the partial output as input, which counts as new input tokens. With the aggregator's pricing (official price × 1.3 for users), you pay for those extra input tokens, but you save on output tokens that would have been regenerated. For long responses, the savings can be significant.

Key contributors earn credits at official price × 1.1 (or × 1.2 for premium models) in USDC, so efficient token usage benefits the ecosystem.

Best practices

• Prefer models with native resume (OpenAI Responses API) for long-running or critical streams.
• Always persist partial output so you can fall back to a continuation request.
• Validate that the continuation is coherent—some models may repeat or contradict earlier text.
• Monitor resume success rates per backend to choose the right model for each task.

By understanding which backends support true resume and which require workarounds, you can build resilient streaming clients that minimize wasted tokens and deliver a smoother experience.