Decoding Aggregator Error Envelopes: Which Model, Which Layer Failed?

When you send a request to an LLM API aggregator, errors can originate from various layers: the aggregator's infrastructure, the upstream model provider, or even your own request. Understanding the error envelope—the structured error response—helps you pinpoint the source quickly. This guide explains the common layers and how to identify which component failed.

The Layers of an Aggregator Request

A typical request passes through several layers:

• Client: Your application or script.
• Aggregator: The service that routes your request, manages authentication, and handles billing.
• Model Provider: The upstream API that actually runs the model (e.g., Anthropic, OpenAI, DeepSeek).
• Model: The specific model instance processing your input.

Errors can occur at any layer, and the error envelope usually includes clues about where things went wrong.

Anatomy of an Error Envelope

Most aggregators return errors in a JSON object with fields like:

• error: Contains message, type, code, and sometimes param.
• provider: Indicates the upstream provider if the error came from there.
• request_id: A unique identifier for tracing.
• layer: Sometimes explicitly states where the error occurred (e.g., "aggregator", "provider").

Not all aggregators include every field, but the structure is similar. The key is to look for indicators of origin.

Identifying Model-Side Failures

Model-side errors originate from the upstream provider or the model itself. Signs include:

• Provider-specific error codes: For example, OpenAI's insufficientquota or Anthropic's overloadederror.
• HTTP status codes like 429 (rate limit) or 503 (service unavailable) from the provider.
• Error messages mentioning the model name or provider-specific terminology.
• provider field present in the envelope.

These errors mean the aggregator successfully routed your request, but the provider or model could not fulfill it. Common causes: rate limits, model overload, invalid parameters for that model, or provider outages.

Identifying Aggregator-Side Failures

Aggregator-side errors occur before or after the request reaches the provider. Signs include:

• Authentication errors (e.g., invalid API key) returned by the aggregator.
• Billing or quota errors related to your aggregator account.
• Routing errors (e.g., model not supported, no available providers).
• HTTP status codes like 401 (unauthorized) or 402 (payment required) from the aggregator.
• Error messages mentioning the aggregator's name or generic terms.

These errors indicate the aggregator itself could not process your request, often due to configuration, account, or infrastructure issues.

Practical Debugging Steps

1. Check the HTTP status code: 4xx usually indicates a client or aggregator issue; 5xx suggests provider or aggregator server problems.
2. Inspect the error.type and error.code: Map them to known provider or aggregator codes.
3. Look for a provider field: If present, the error likely came from the upstream provider.
4. Examine the message: Provider messages often include model names or provider-specific details.
5. Use the request_id: Contact support with this ID if you suspect an aggregator issue.

Example Scenarios

• Scenario A: You get a 429 error with provider: "openai" and message about rate limits. This is a model-side failure; retry with backoff or switch models.
• Scenario B: You get a 401 error with no provider field and message about invalid API key. This is an aggregator-side failure; check your API key.
• Scenario C: You get a 400 error with param: "max_tokens" and a message about exceeding context length. This could be either side, but often the aggregator validates parameters before sending to the provider. Check the model's context window.

Best Practices for Handling Errors

• Implement retries with exponential backoff for transient provider errors (429, 503).
• Log the full error envelope for debugging.
• Fallback to alternative models if a provider is down.
• Monitor aggregator status pages for known incidents.
• Validate requests client-side to avoid common 400 errors.

By understanding the layered error envelope, you can quickly diagnose whether the issue lies with your request, the aggregator, or the upstream model provider. This reduces downtime and improves the reliability of your LLM-powered applications.