Replaying a Production Call From the Request Log on One Key: Reconstructing Prompt, Params and Backend

Why faithful replay matters

When a call behaves oddly in production, an approximate re-run rarely reproduces the problem. Small differences in the prompt text, sampling parameters, or the backend that served the request can change the output. The goal of a replay is to change exactly one thing at a time, starting from a faithful copy of the original call.

The aggregator's request log is the source of truth for that reconstruction. Because you call many models (Claude, GPT, DeepSeek, Qwen, GLM, Kimi) with one API key and pay the official price × 1.3 in USDC, the log also makes it easy to see which call you are re-running and what it costs to do so.

What the request log should give you

Treat the log entry as a record with distinct layers. Before you replay anything, confirm you can recover each of these for the call in question:

• Identity and timing: request ID, timestamp, and the key or project the call was made under.
• Backend route: which model actually served the request, including any routing or fallback that occurred.
• Prompt payload: system message, user message, and any assistant or tool turns, as sent.
• Sampling parameters: temperature, top-p, max tokens, stop sequences, and any model-specific options.
• Response and status: the returned text, finish reason, token usage, and any error or warning fields.

If a field is not visible in your log view, check whether it is stored but hidden behind a detail view rather than missing. Reconstructing a call is only as faithful as the fields you can read.

Step 1: Capture the original request

Open the specific log entry for the odd call and copy the fields rather than retyping them. Save the reconstructed request as a file (for example, a JSON body plus a small script) so the replay is repeatable and diffable. Keep the original request ID alongside it; you will want to reference it when you compare outputs.

Step 2: Reconstruct the prompt exactly

Rebuild the message array in the same order and with the same roles as the original. Watch for details that are easy to lose:

• Whitespace, newlines, and formatting inside the prompt can matter.
• System messages are often prepended by your application; confirm the logged prompt includes them rather than reconstructing them from memory.
• If tools or function schemas were sent, reproduce them verbatim — schema differences can change behaviour.
• If the request was truncated or assembled from a template, reproduce the assembled result, not the template inputs.

When in doubt, copy the logged payload directly into your replay harness.

Step 3: Restore the sampling parameters

Parameters are the most common source of unreproducible behaviour. Copy every value from the log, including defaults. Pay attention to:

• Temperature and top-p, which change the shape of sampling.
• Maximum output tokens and any stop sequences.
• Any provider-specific options that the aggregator passes through.

If the original request relied on defaults, replay with the same defaults on the same model route rather than supplying your own values. Documenting the parameter set next to the request makes later comparisons meaningful.

Step 4: Confirm the backend

The same prompt and parameters can behave differently across model versions, providers, or regions. Use the log to identify which backend served the original call, and route the replay to that same backend where the aggregator allows it. If the log records a fallback or retry, note that the response you saw may have come from a secondary route; replay against the route that actually produced the output.

Step 5: Re-issue without touching production

Replay from a separate harness or a scratch key rather than through the production path. Send the reconstructed request as a single call, capture the full response, and compare it to the logged response field by field: text, finish reason, and usage. If they diverge, change one variable at a time — first confirm determinism with the same parameters, then vary a single parameter to test a hypothesis.

Because you top up in USDC and pay official price × 1.3 per call, keep replays deliberate: isolated single calls are cheaper to reason about than replaying a batch.

Step 6: Compare and iterate

Keep a short log of each replay attempt: what you changed, what you observed, and the request ID of the replay. This turns debugging into a controlled experiment. When you find the variable responsible, you can fix the caller and re-run the original scenario once to confirm.

Common reconstruction pitfalls

• Retyped prompts: small edits silently change the input. Copy from the log.
• Missing system messages: the application often adds them outside your visible template.
• Assumed defaults: a parameter left unset may not equal the value you would choose yourself.
• Wrong backend: replaying against a different model version can mask or mimic the original issue.
• Multi-turn context: if the call depended on earlier turns, reconstruct the full conversation, not just the last user message.

Operational habits that make replay easy

• Log the full request payload, not just the response.
• Record the resolved backend and any fallback in the log.
• Tag production calls with a trace or feature identifier so you can find them later.
• Store replay scripts with the original request IDs so investigations are reproducible.

With one key and a complete request log, a faithful replay is mostly disciplined copying: reconstruct the prompt, restore the parameters, pin the backend, and change one thing at a time.