Where the Request Body Stops Being Yours: Prompt vs Metadata in Aggregator Logs

Introduction

When you send a request to an LLM API aggregator, your request body contains two fundamentally different types of information: the prompt itself (the text you want the model to process) and the metadata (everything else that makes the request work). Understanding where the boundary lies – and how aggregators handle each part – is essential for making informed decisions about privacy, security, and debugging.

What Is Prompt Content?

Prompt content is the semantic payload: the messages, system instructions, user inputs, and any other text that directly influences the model's output. This is the part you own intellectually and often the part you care most about protecting.

• System prompts – instructions that set the model's behavior.
• User messages – the actual queries or inputs from end users.
• Assistant messages – prior model responses included for context.
• Tool definitions and function calls – structured schemas that tell the model what actions it can take.

From the aggregator's perspective, prompt content is opaque data that must be forwarded to the upstream provider to fulfill the request. Most aggregators treat it as transient – processed in memory and not persisted beyond what is necessary for billing and abuse prevention.

What Is Request Metadata?

Request metadata is everything else in the request body or headers that describes how the request should be handled, rather than what should be processed.

• Model identifier – which model to route to (e.g., a specific Claude, GPT, or DeepSeek variant).
• Parameters – temperature, topp, maxtokens, stop sequences, and other generation controls.
• User identifiers – an end-user ID, account ID, or API key identifier.
• Routing hints – region, latency preferences, or fallback order.
• Timestamps and request IDs – for tracing and idempotency.

Metadata is often logged more liberally because it is small, structured, and useful for operational purposes like monitoring, billing, and debugging. But that convenience can create privacy risks if metadata contains sensitive information.

Where the Boundary Blurs

In practice, the line between prompt and metadata is not always clear.

• User IDs in metadata – If you pass an end-user identifier in a header or a dedicated field, that identifier is metadata, not prompt. It can be logged and correlated across requests.
• Prompt-derived metadata – Some aggregators extract token counts, detected languages, or content categories from the prompt itself. Those derived values are metadata, even though they originated from prompt content.
• Tool schemas – Function definitions are technically prompt content (they are sent to the model), but aggregators may also parse them as metadata for validation or routing.

This blurring means that even if an aggregator promises not to log prompt content, it may still retain information that reveals something about your prompts.

Why the Distinction Matters

Privacy

Prompt content often contains proprietary business logic, personal data, or confidential information. Metadata, while less rich, can still be sensitive – for example, if user IDs are email addresses or if model selection reveals business strategy.

Debugging

When something goes wrong, metadata logs help you trace the request path: which model was used, what parameters were set, and when the request occurred. Prompt logs would help you reproduce the exact input, but they are often unavailable for privacy reasons.

Compliance

Regulations like GDPR or CCPA may treat prompt content and metadata differently. Metadata that includes personal data (e.g., user IDs) is subject to the same requirements as prompt content if it can be linked to an individual.

How Aggregators Handle Logs

Aggregators vary in their logging policies. Some common patterns include:

• No prompt logging – Prompt content is never written to disk; only metadata is logged.
• Short-term prompt logging – Prompt content is kept for a limited time (e.g., for abuse detection) and then deleted.
• Opt-in logging – Users can choose to enable prompt logging for debugging, usually with clear warnings.
• Metadata-only logging – Metadata is logged by default, often with configurable retention.

Because policies differ, it is important to read the documentation of any aggregator you use and understand what is logged, for how long, and who can access it.

Practical Strategies for Minimizing Metadata Exposure

You cannot control an aggregator's internal logging, but you can reduce the sensitivity of the metadata you send.

• Avoid sending direct personal identifiers in metadata fields. Use pseudonymous IDs that are meaningless outside your system.
• Do not put sensitive data in headers like custom user-agent strings or referer fields.
• Use per-request unique IDs that cannot be correlated across sessions if correlation is not needed.
• Review tool schemas to ensure they do not embed secrets or internal URLs.
• Prefer aggregators with clear, auditable logging policies and, where available, configurable retention.

Conceptual Example: A Request Breakdown

Consider a request to an aggregator with a single API key.

• Prompt content: "Summarize the following customer feedback: ..."
• Metadata: model: "claude-3-sonnet", temperature: 0.7, user: "user12345", requestid: "req_abc123".

If the aggregator logs only metadata, it will record that user_12345 made a request to claude-3-sonnet at a certain time, but not the customer feedback text. If the aggregator logs prompt content, it will also store the feedback text, which may contain personal data.

Conclusion

Knowing where the request body stops being yours – where prompt content ends and metadata begins – helps you ask the right questions when choosing an aggregator. While you can never fully control what happens on the provider side, you can minimize risk by sending only necessary metadata, using pseudonymous identifiers, and favoring aggregators with transparent logging practices. Always review the specific policies of any service you use, and design your application to treat both prompt and metadata as potentially observable.