What the Aggregator Actually Logs on a Single Key: Mapping Prompt and Response Retention Before You Send Anything Sensitive

Why the request path matters

When you call a model through an aggregator, your request does not go straight to the model vendor. It passes through an intermediate service first. That means a single request can produce artifacts in more than one place, and each place may have its own retention behavior.

Before sending anything sensitive, it helps to map the path: what leaves your client, what the aggregator holds, and what the upstream vendor receives.

The two layers to think about

• Your client layer. The code or tool making the call. You control what it logs locally.
• The aggregator layer. The service that accepts your key, routes the request, and returns the response.
• The upstream vendor layer. The model provider that actually runs inference.

A request touches all three. The aggregator sits in the middle, so it can observe both the outbound request and the inbound response.

Artifacts a request can leave behind

Think in terms of artifact categories rather than exact fields, since implementations differ.

• Prompt content. The messages you send. The aggregator must read these to route and forward them, so assume it can see them in transit.
• Completion content. The model's output. It passes back through the aggregator to reach you.
• Request metadata. Model identifier, token counts, parameters such as temperature, and streaming flags.
• Headers. Authentication headers, content type, client identifiers, and any custom headers your client sets.
• Timestamps and timing. When the request arrived, how long it took, and status codes.
• Accounting records. Usage tied to your key for billing and credit purposes.

What is logged for retention versus processed in memory and discarded is a policy question, not a technical certainty. Ask rather than assume.

Aggregator layer vs. upstream vendor layer

These are separate retention domains.

• The aggregator necessarily handles your key, the prompt, and the response to route traffic and meter usage. Its retention policy governs how long it keeps request or response content, if at all.
• The upstream vendor receives a request forwarded on your behalf. Its own retention and training policies apply to that data.

A common misconception is that using an aggregator hides your content from the vendor. It does not. The aggregator is an additional hop, not a substitute for the vendor relationship.

Reasoning about retention on one key

A few practical habits:

• Separate identity from content. Your key identifies the account. Content is a separate question. Knowing who made a request is not the same as knowing whether the content is stored.
• Assume transport visibility. Any intermediary that forwards a prompt can technically read it. Decide what you are comfortable sending through any intermediary.
• Check the billing trail. Usage and credit records tied to a key are typically retained for accounting. These are metadata, not necessarily prompt text, but confirm the boundary.
• Treat retention as a policy question. Ask what is stored, for how long, and for what purpose. Do not infer from the API shape alone.

Reducing what you send

You often control how much sensitive material enters the request path.

• Redact or tokenize identifiers before they reach the API.
• Send only the context the task needs rather than full documents.
• Keep secrets out of prompts; pass configuration through your own systems.
• Log less on the client side, since your own logs are another retention surface.

A pre-send checklist

• What content am I sending, and would I be comfortable if an intermediary could read it?
• Which layer am I relying on for privacy, and does that assumption hold for both the aggregator and the vendor?
• What does my own client log, and how long does it keep it?
• Is the sensitive part necessary for the task, or can it be redacted?

The takeaway

A single key request can leave artifacts at the aggregator layer and the upstream vendor layer, plus whatever your client records. Treat retention as something to verify per layer, minimize what you send, and never assume the middle hop removes the vendor's own policies from the picture.