Routing by Task Type on One Key: Sending Classification, Summarizing and Code Work to Different Backends

Why route by task instead of by model name

Most teams pick one "best" model and send everything to it. That works until you notice that a large share of your traffic is short, simple work — tagging a support ticket, extracting a field, classifying intent — that does not need a frontier reasoning model. Routing by task type lets you match capability to cost and latency.

The constraint that usually blocks this is key management: each provider wants its own account, key, and billing. An aggregator that exposes many models behind one API key removes that friction, so routing becomes a config decision rather than an integration project.

The decision table

Use task shape, not brand recognition, as the primary signal.

| Task type | Typical signals | What to optimize for | Backend class to prefer |
|---|---|---|---|
| Classification / tagging | Short input, fixed label set, high volume, strict output format | Latency and cost per call | Small, fast, inexpensive general models |
| Extraction / structured output | Input needs parsing into JSON or fields | Instruction following and format reliability | Mid-tier models with strong schema adherence |
| Summarization | Medium-to-long input, faithful compression | Context handling and instruction fidelity | Mid-tier models; long-context variants for big inputs |
| Long-form reasoning | Multi-step logic, planning, ambiguous problems | Reasoning depth | Frontier reasoning-capable models |
| Code generation / review | Diffs, functions, tests, refactors | Code competence and tool use | Code-strong models from any major family |
| Conversational UX | Open-ended chat, tone-sensitive | Fluency and steering | Mid-tier or frontier, depending on stakes |

The rows are deliberately about work shape. A model that is famous for chat may be a poor fit for strict JSON extraction, and a cheaper model may beat it on a label task.

Cheap classification: the highest-leverage switch

Classification is where routing pays off most, because volume is high and the task is bounded.

• Keep the label set small and closed. Enumerate allowed labels in the system prompt.
• Ask for the label only, or a tiny JSON object. Long explanations cost tokens and invite drift.
• Validate the output against your label set; retry once on a miss, then fall back to a stronger backend.
• Batch or queue where latency is not user-facing.

A useful rule: if a human reviewer could do the task in a couple of seconds from the input alone, a small model is usually the right first attempt. Escalate only on validation failure.

Summarization: watch input size, not just model tier

Summarization is dominated by input length. Two things matter more than the model's reputation:

• Context fit. Chunk long documents deliberately rather than hoping truncation is harmless.
• Faithfulness instructions. State what must be preserved (names, numbers, decisions) and what may be dropped.

For routine internal summaries, a mid-tier model is usually enough. Reserve frontier models for summaries where an error is costly — contract review summaries, incident timelines — and even then, treat output as a draft.

Long-form reasoning: spend here

Multi-step reasoning, planning, and ambiguous problem decomposition are where cheap routing backfires. A small model that produces a plausible but wrong plan costs more in review time than the tokens saved.

• Send genuinely hard tasks to reasoning-capable frontier models.
• Give them room: clear problem statement, constraints, and the expected output shape.
• Do not use them for classification. It is the most expensive way to assign a label.

Code generation: competence over tier

Code tasks have a distinct failure mode: output that looks right but does not compile, or compiles but changes behavior. Choose backends known for code work rather than the general-purpose frontier model by default.

• For generation, ask for the full function plus tests where feasible.
• For review, provide the diff and the surrounding context, not the whole repository.
• Run whatever static checks you have before a human sees the output. Routing cannot substitute for verification.

Implementing routing on one key

Because all backends sit behind a single API key, routing is a lookup in your own code.

1. Tag requests at the call site. Pass a task type (classify, summarize, reason, code) alongside the prompt.
2. Map task type to a model. Keep the mapping in one config file so it can change without touching call sites.
3. Validate where possible. Schema checks for structured output, label checks for classification.
4. Escalate on failure. One retry on a stronger backend, then surface the error.
5. Log task type, model, and token counts. Without this you cannot tell whether routing helped.

Cost model to keep in mind

On this platform you pay the official model price × 1.3 for usage, and key contributors are credited official price × 1.1 (×1.2 for premium contribution) in USDC. The practical consequence: relative differences between backends are preserved, so moving high-volume classification from a frontier model to a small one still saves proportionally. Routing decisions based on official pricing remain valid.

Common mistakes

• Routing by fame. The most talked-about model is rarely the cheapest fit for tagging.
• No output validation. Classification without a label check silently degrades.
• Skipping the escalation path. A cheap-first strategy needs a defined fallback.
• Ignoring latency. A cheap model that is slow under load can be worse for interactive UX than a slightly pricier fast one.
• Never re-evaluating. Model lineups change; revisit the mapping periodically against your own logs.

A starting configuration

• Classification and extraction → small or mid-tier model, strict output validation, escalate on failure.
• Summarization → mid-tier model, chunk long inputs, fidelity instructions.
• Long-form reasoning → frontier reasoning model, reserved for genuinely hard tasks.
• Code → code-strong model, with tests and static checks in the loop.

Start with this table, measure with your logs, and adjust the mapping. The goal is not the cheapest call every time — it is the cheapest call that reliably does the job.