Tool Calls That Stream: Reading tool_call Deltas Inside an SSE Response

Why Tool Calls Stream Differently

When you stream a chat completion, the model doesn't wait to finish a tool call before sending anything. Instead, it emits the call in pieces. For text, that's just chunks you append. For tool calls, it's structural: the response contains a tool_calls array where each entry has an index, an id, a function.name, and a function.arguments field that accumulates as a string.

The first delta for a tool call typically sets the id and name. Subsequent deltas only carry arguments. Each delta's arguments is a fragment—not valid JSON on its own. If you try to JSON.parse the first fragment, you'll get an error because it's something like {"loc instead of a complete object.

The Anatomy of a Tool Call Delta

In an OpenAI-compatible SSE stream, tool call deltas appear inside choices[0].delta. A typical sequence looks like this:

• Delta 1: delta.toolcalls[0] = { index: 0, id: "callabc", function: { name: "get_weather", arguments: "" } }
• Delta 2: delta.tool_calls[0] = { index: 0, function: { arguments: "{\"ci" } }
• Delta 3: delta.tool_calls[0] = { index: 0, function: { arguments: "ty\": \"San" } }
• Delta 4: delta.tool_calls[0] = { index: 0, function: { arguments: " Francisco\"}" } }

Some providers include the id and name only in the first delta; others repeat them. Always check index to know which tool call a fragment belongs to. You may have multiple tool calls interleaved, so you need a buffer per index.

Why Naive Parsing Breaks

The arguments string is not valid JSON until the model has emitted the closing brace. Parsing a fragment fails for three reasons:

• Incomplete structure: you have an opening brace but no closing brace.
• Truncated strings: a string value may be cut mid-character, e.g., "San missing the closing quote.
• Escape sequences: a backslash or Unicode escape might be split across deltas.

Even if a fragment happens to be valid JSON, it's likely just a partial object—missing keys that arrive later. Parsing early gives you an incomplete object that you can't reliably use.

Buffering and Reassembly Pattern

The solution is to accumulate all fragments for each tool call and parse only once the stream ends. Here's a conceptual pattern:

1. Initialize an empty map: toolCalls = new Map().
2. For each SSE event, extract delta.tool_calls (if present).
3. For each tool call delta, get its index.
4. If the map doesn't have that index, create an entry: { id: '', name: '', arguments: '' }.
5. Update the entry: if id is present, set it; if function.name is present, append or set it; always append function.arguments to the existing string.
6. After the stream ends, iterate over the map and JSON.parse each arguments string.

This ensures you only parse a complete JSON document. The final arguments string is the concatenation of all fragments, which forms a valid JSON object (assuming the model behaved correctly).

Handling Edge Cases

• Multiple tool calls: the index field distinguishes them. Never assume there's only one.
• Missing id or name in later deltas: keep the values from the first delta that provided them.
• Empty arguments: some tool calls have no parameters, so arguments may be an empty string. Treat that as an empty object {}.
• Malformed JSON: if parsing fails, log the raw string for debugging. It could be a model error or a stream interruption.

Why This Matters for Aggregators

If you're using an API aggregator that routes to multiple model providers, you'll see variations in how tool call deltas are structured. Some providers might send the full arguments in one delta; others fragment more aggressively. A robust buffering approach works regardless of the provider, so you can switch models without rewriting your stream handler.

Remember that billing is based on the official price multiplied by 1.3 for users, and contributors earn back a multiple of the official price (1.1 for standard, 1.2 for premium) in USDC. Streaming doesn't change the token accounting—you still pay for what you use.

Summary

Tool call arguments arrive as fragments, so you must buffer them per tool call index, concatenate the arguments string, and parse only after the stream completes. This pattern avoids partial JSON errors and works across providers.