What Fields Land in the Request Log of an Aggregated LLM Call

When you send a request through an aggregated LLM API, the platform records details to help you track usage, debug issues, and manage costs. The exact field names and structure vary by provider, but most request logs contain similar categories of information. This article explains the typical fields you might see and how to interpret them.

Request Metadata

Every logged call starts with basic identifiers and timing information.

• Request ID: A unique identifier for the call, useful for tracing and support inquiries.
• Timestamp: When the request was received and completed, often in UTC.
• User ID or API key ID: Identifies which account or key made the request (often hashed for privacy).
• Endpoint: The specific API route called, such as /v1/chat/completions.
• HTTP method: Usually POST for model calls.
• Status code: The HTTP response status (e.g., 200 for success, 4xx or 5xx for errors).
• Latency: Time taken to process the request, sometimes split into queue time and inference time.

Model and Routing Details

Aggregators often route requests to different providers or model versions. The log captures these decisions.

• Requested model: The model name you specified in your request (e.g., claude-3-opus).
• Resolved model: The actual model and version that handled the request, which may differ if the aggregator routes to a fallback or a specific deployment.
• Provider: The underlying infrastructure provider (e.g., Anthropic, OpenAI, DeepSeek).
• Region: The geographic region where the model ran, if applicable.
• Routing reason: Why a particular provider or model was chosen (e.g., load balancing, fallback, cost optimization).

Input and Output Parameters

These fields describe the content and settings of your request and the model's response.

• Prompt tokens: Number of tokens in the input (your prompt).
• Completion tokens: Number of tokens in the model's output.
• Total tokens: Sum of prompt and completion tokens.
• Temperature, topp, maxtokens: Sampling parameters used.
• Stop sequences: Any stop sequences provided.
• System prompt: The system message, if included (may be truncated or hashed for privacy).
• Message count: Number of messages in a chat completion request.
• Response ID: A unique identifier for the model's response.
• Response content: The actual text generated (may be omitted or redacted depending on privacy settings).

Usage and Billing Metrics

Aggregators track usage to calculate costs and credits.

• Input cost: Calculated as official price × 1.3 for users; for contributors, credits are based on official price × 1.1 (or × 1.2 for premium models).
• Output cost: Similarly calculated based on output tokens.
• Total cost: Sum of input and output costs.
• Currency: Usually USDC.
• Billing status: Whether the call was billed successfully.
• Credit applied: Any credits or discounts applied to the call.

Error and Debugging Information

When something goes wrong, logs include details to help you diagnose.

• Error code: A machine-readable code (e.g., ratelimitexceeded, invalid_request).
• Error message: A human-readable description.
• Retry count: Number of retries attempted by the aggregator.
• Upstream error: If the provider returned an error, details may be included.
• Request body: The raw request payload (often truncated or redacted for sensitive data).
• Response body: The raw response from the provider (may be truncated).

Privacy and Compliance Fields

Depending on your settings and local regulations, logs may include flags related to data handling.

• Data retention policy: How long the log is stored.
• Redaction status: Whether sensitive fields were redacted.
• Consent flags: If user consent was recorded for data processing.
• Anonymization: Whether identifiers were anonymized.

How to Use Request Logs

Request logs are valuable for several purposes:

• Debugging: Trace failures by request ID and inspect error details.
• Cost monitoring: Track token usage and costs per model or user.
• Performance tuning: Analyze latency and adjust routing or parameters.
• Auditing: Maintain records for compliance or internal review.
• Optimization: Identify expensive patterns and switch to more cost-effective models.

Conclusion

The exact fields in a request log depend on your aggregator and configuration, but most include request metadata, model routing, token usage, billing details, and error information. By understanding these fields, you can better manage your LLM integrations, control costs, and troubleshoot issues effectively. Always refer to your provider's documentation for the precise schema and privacy considerations.