Per-Key or Per-Model? Where the Rate Limit Actually Applies on One Aggregator Key

The Layered Reality of One Key

When you use a single API key from an aggregator to access models from multiple vendors, that key is not a direct line to each provider. Instead, it fans out through the aggregator's infrastructure to six different backends. Each backend may enforce its own rate limits. As a result, a 429 Too Many Requests error can originate from several distinct layers.

Layer 1: Aggregator-Level Limits

The aggregator itself may impose limits on your key. These could be global across all models, or scoped to specific models or endpoints. For example, the aggregator might limit the total number of requests per minute for your key regardless of which model you call. If you exceed that, you'll get a 429 from the aggregator before any upstream provider is contacted.

Layer 2: Upstream Provider Limits

Each model vendor (e.g., OpenAI, Anthropic, DeepSeek) applies its own rate limits. These are typically based on requests per minute (RPM) and tokens per minute (TPM). Since the aggregator likely uses shared pools of upstream accounts, your requests may be subject to limits that are shared with other users. When you hit an upstream limit, the provider returns a 429, which the aggregator forwards to you.

Layer 3: Per-Model Limits

Even within a single provider, different models can have different rate limits. For instance, a flagship model might have lower RPM than a smaller, cheaper model. The aggregator may also impose per-model limits to prevent abuse or manage costs. So a 429 might be specific to the model you're calling, not your overall usage.

How to Tell Which Layer Returned a 429

When you receive a 429, the response often includes headers or a body that can hint at the source. Look for:

• Error message details: The aggregator may include a message like "upstream rate limit exceeded" or "aggregator rate limit exceeded." Read carefully.
• Headers: Check for Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining, or custom headers prefixed with X-Aggregator- or X-Provider-. These can indicate which limit was hit.
• Timing: If the error occurs immediately without any processing delay, it's likely an aggregator-level limit. If it happens after some processing time, it might be upstream.
• Model specificity: If the error only occurs for one model, it's likely a per-model or upstream limit for that model.

Practical Steps to Diagnose and Mitigate

• Log everything: Record the full response, including headers, for each 429. Patterns will emerge.
• Test with a single model: If you get 429s across all models, the issue is likely aggregator-level. If only one model fails, it's per-model or upstream.
• Check aggregator documentation: The aggregator may document its rate limits and how they interact with upstream limits.
• Implement exponential backoff: When you get a 429, retry with increasing delays. This helps with transient upstream limits.
• Distribute load: If possible, spread requests across multiple models or providers to avoid hitting any single limit.
• Contact support: If you consistently hit limits despite low usage, the aggregator may be able to adjust your limits or explain shared pool constraints.

Conclusion

Understanding the layered nature of rate limits on an aggregator key is crucial for building robust applications. By recognizing that a 429 can come from the aggregator, an upstream provider, or a specific model, you can diagnose issues faster and implement appropriate retry and load-balancing strategies.