Assembling Streamed Tool Call Arguments When They Arrive in Fragments Across the SSE Stream

Why tool call arguments arrive in fragments

When a model supports tool (function) calling and you enable streaming, the API typically sends the tool call as a series of incremental events rather than one finished object. The arguments payload is a JSON string that the model emits token by token, so a single logical call may be spread across many SSE chunks.

The practical consequence: at any moment mid-stream, the accumulated string is usually invalid JSON. You must not parse it on every chunk. Instead you buffer it and parse once at the end.

Two fields tend to be involved:

• Identity and metadata — an index, an id, and the function name. These may appear once or be repeated.
• Arguments — a string that grows as chunks arrive.

The accumulate-versus-replace distinction

The most common bug is treating every field the same way. Ask, per field: is this a full value or a partial one?

• Concatenate (append) fields that are emitted incrementally. The arguments string is the canonical example: each chunk carries the next slice, so you append.
• Replace (set or overwrite) fields that are emitted as complete values. The call id, the function name, and the type are usually sent whole. If a later chunk repeats them, overwriting with the same value is harmless; concatenating would corrupt them.
• Ignore (or treat as no-op) fields that are empty strings or absent. Many chunks include an empty arguments fragment; appending an empty string is fine, but skipping is cheaper.

A useful mental model: the arguments string is a byte stream you are reassembling, while the metadata is a small object you are filling in.

Identifying which call a fragment belongs to

A single response can contain more than one tool call, and their fragments may interleave. Do not assume a flat append-only log.

• Use the index field to route each fragment to the right accumulator. If your provider doesn't send an index, fall back to the call id once it appears.
• Maintain a map keyed by index (or id) where each entry holds the id, name, and an arguments buffer.
• If fragments for one call are guaranteed to be contiguous, a single active accumulator works, but the map is safer and costs little.

Do not rely on arrival order alone to decide identity; a name fragment arriving after an arguments fragment is still the same call.

Deciding when a call is complete

Completion is signalled differently across APIs, so combine signals rather than relying on one:

• Explicit finish reason. A chunk may carry a finishreason such as toolcalls (or a stop reason of the same kind). That is the strongest signal that the model has finished emitting calls.
• Stream termination. The SSE stream ends with a sentinel event (often [DONE]) or a closed connection. At that point, flush all accumulators.
• Per-call boundaries. Some APIs emit a discrete event when a specific tool call's arguments are finalized. If present, that lets you dispatch each call as soon as it closes rather than waiting for the whole response.

A robust pattern is: parse and execute only on an explicit completion signal, with stream end as a fallback flush. Never execute mid-stream just because the accumulated string happens to parse — a valid prefix can occur by accident (for example, {} before the real fields arrive).

A reference accumulation loop

At a conceptual level:

1. Open the SSE connection and read events line by line.
2. For each event, decode the JSON payload and inspect the delta.
3. If a tool-call delta is present, look up the accumulator by index (create it if new).
4. Set id and name if provided; append the arguments fragment if non-empty.
5. On a finish signal for tool calls, stop appending.
6. After the stream ends, for each accumulator, parse the buffered arguments string as JSON.
7. Handle parse failure as a recoverable error (retry, ask the model to re-emit, or surface it), then invoke the tool with the parsed object and the recorded name.

Keep raw buffers until parsing succeeds; log a truncated form for debugging rather than dumping entire payloads.

Validation before execution

Accumulated arguments are model output, not trusted input.

• Parse the complete string as JSON; treat malformed JSON as a failed call, not a partial success.
• Check the parsed value against the tool's schema — required fields, types, and enums.
• Confirm the function name matches a tool you actually exposed; reject unknown names.
• Bound the buffer size so a runaway stream cannot exhaust memory.

If validation fails, decide deliberately: surface the error to the user, retry the turn, or feed the validation error back to the model for a corrected call.

Operational tips

• Grouping keeps cost predictable. One API key across many models means you can choose a cheaper model for tool-heavy loops and a stronger one for reasoning, then pay per call at the published multiplier. Contributors who help supply keys are credited back in USDC, but that doesn't change the per-call arithmetic.
• Keep the SSE parser strict. Handle multi-line data: fields and comment lines; a partial line at a buffer boundary is a common source of lost fragments.
• Make execution idempotent. If you retry a turn, deduplicate tool calls by id so a side-effecting tool doesn't run twice.
• Bound concurrency. Parallel tool calls still map to the same response; process them in the order you choose, not the order fragments happened to arrive.

Checklist

• Route fragments by index/id, not by arrival order.
• Append arguments; set or overwrite metadata.
• Parse once, after an explicit completion signal or stream end.
• Validate against the tool schema before invoking.
• Handle malformed JSON and unknown tool names as first-class errors.