Prompt Caching Across Six Backends on One Key: Who Bills You Again for the Same Prefix
Prompt caching lets backends reuse a repeated prefix so you aren't billed again for the same tokens. This guide explains how caching works conceptually, why it matters even more when a single API key fronts multiple models, and how to design prompts and routing to maximize cache hits—without relying on provider‑specific numbers.
Why repeated prefixes cost money
Every chat completion resends the whole conversation. If your system prompt is 2,000 tokens, you pay for those 2,000 tokens on each call—unless the backend supports prompt caching. Caching stores the processed prefix so later calls can skip recomputing it, reducing both cost and latency.
How caching works (conceptually)
- Prefix matching: Backends that offer prompt caching typically look for an exact prefix match. If the first N tokens are identical across requests, they can be reused. Any change—even a single character—breaks the match.
- Explicit vs. implicit: Some APIs require you to mark cacheable sections explicitly; others detect repeated prefixes automatically. Support varies by model and provider.
- Cache lifetime: Cached entries expire after a period determined by the provider, often minutes to hours. High request frequency within that window maximizes savings.
- Cost model: Providers usually charge a reduced rate for cached tokens (sometimes a write fee, sometimes just a lower read fee). The exact pricing is set by each backend.
Why a single key across six backends changes the game
When you call Claude, GPT, DeepSeek, Qwen, GLM, and Kimi through one aggregator key, caching becomes a routing decision. Each backend has its own caching rules. A prefix cached for one model does not carry over to another. So the question shifts from “does this model cache?” to “which model will I call repeatedly with the same prefix?”
- Stick with one model for repetitive tasks: Switching models between calls destroys any cache benefit. Keep a given workflow on the same backend if the prefix is long.
- Group similar requests: Batch requests that share a long system prompt or few‑shot examples so they hit the same cache.
- Be mindful of fallbacks: Automatic failover to a different model may bypass the cache. Understand your routing logic.
Designing prompts for cache hits
Put stable content first:
- System instructions and tool definitions at the very top.
- Static examples before dynamic user input.
- Long documents (if reused) before the question.
Avoid inserting timestamps, random IDs, or user‑specific greetings at the beginning—they break prefix matching for everyone. Instead, place variable content at the end.
What about pricing?
On this aggregator, you pay the official price × 1.3 for all usage, including cached tokens (if the backend charges for them). Caching reduces the number of tokens you send, so your effective spend drops—but the multiplier still applies. Contributors who refer others receive official price × 1.1 (or × 1.2 for premium) credited in USDC, independent of caching.
Practical checklist
- Check which of your target models support prompt caching; consult each provider’s docs.
- Keep prefixes byte‑identical across calls—avoid invisible whitespace changes.
- Route repeated tasks to the same backend; use model switching only when necessary.
- Monitor your usage: a drop in prompt tokens after enabling caching is a good sign.
- Remember that cache lifetimes vary; high‑frequency workloads benefit most.
Caching is not automatic magic—it rewards deliberate prompt structure and consistent routing. With one key across six backends, the main risk is scattering similar requests across models and losing the cache. Consolidate where it counts.