Where Token Counts Show Up in a Streamed Response: Reconstructing Usage From Chunks

Why Token Counts Matter in Streaming

When you stream a response from an LLM API, the final token usage is not always presented in a single, consistent place. Different backends (e.g., OpenAI, Anthropic, or open-source models) embed usage data in varying chunk types or fields. For accurate billing and usage tracking, you need to reconstruct the total token counts from these streamed chunks and reconcile them against your USDC deduction.

Common Patterns for Usage in Streams

Usage information typically appears in one of three ways:

• Final chunk with usage field: Some APIs include a usage object in the last chunk (or a dedicated usage chunk) that contains prompttokens, completiontokens, and total_tokens.
• Incremental counts per chunk: Less common, but some providers attach token counts to each chunk (e.g., delta with token counts).
• No explicit usage: Some backends omit usage entirely in streaming mode; you must estimate from the text or rely on the API's non-streaming endpoint for exact counts.

Accumulating Token Counts Client-Side

To reconstruct usage, follow these steps:

1. Initialize counters for prompt tokens, completion tokens, and total tokens (set to zero).
2. Parse each chunk as it arrives. Look for a usage field or similar. If present, update your counters (typically the final chunk contains the complete usage).
3. Handle incremental updates: If usage is provided incrementally, add the deltas to your counters.
4. Fallback estimation: If no usage is provided, you can approximate completion tokens by counting characters or using a tokenizer library, but this may be imprecise. For billing reconciliation, prefer exact counts from the API when available.

Example (pseudocode):

prompt_tokens = 0
completion_tokens = 0
for chunk in stream:
    if 'usage' in chunk:
        usage = chunk['usage']
        prompt_tokens = usage.get('prompt_tokens', prompt_tokens)
        completion_tokens = usage.get('completion_tokens', completion_tokens)
    # or handle incremental deltas if provided
total_tokens = prompt_tokens + completion_tokens

Reconciling Against USDC Deduction

On our platform, you pay the official price multiplied by 1.3 in USDC. To verify your deduction:

• Calculate expected cost: Multiply the official per-token price by your total tokens, then apply the 1.3 multiplier.
• Compare with actual deduction: Check the USDC amount deducted from your balance after the request completes.
• Account for rounding: Small discrepancies may occur due to rounding; ensure your calculation matches the platform's rounding rules.

If you are a key contributor, you receive credits at official price × 1.1 (or × 1.2 for premium models) in USDC, so your net cost is effectively lower. However, always base your reconciliation on the actual deduction shown in your account.

Best Practices

• Log usage per request: Store prompt and completion tokens for auditing.
• Use non-streaming for exact counts: If precise billing is critical, consider making a non-streaming call or a separate token-count endpoint if available.
• Monitor for missing usage: Some backends may not include usage in streams; fall back to estimation or non-streaming calls.
• Stay updated: API behavior can change; regularly verify how usage appears in streams.

By understanding where token counts appear in streamed responses and accumulating them correctly, you can ensure accurate cost tracking and reconciliation with your USDC deductions.