Same Task, Two Backends on One Key: Comparing USDC Cost Per Completed Task, Not Per Token

Why cost per task beats cost per token

Token prices are easy to compare but misleading. A cheaper model may need more tokens, more retries, or longer outputs to finish the same job. The real cost is the USDC you spend to get a completed task—including all attempts.

Prerequisites

• An API key from the aggregator (one key works for all supported models).
• A USDC balance on Base (top up without KYC).
• A repeatable task with a clear success condition.

Step 1: Define a task and success criteria

Pick a task you run often, e.g., "summarize a document into 5 bullet points" or "extract fields from an invoice." Define success precisely: output length, required fields, format, and any retry rules. This makes the comparison fair.

Step 2: Run the same task on two backends

Using the same key, send identical inputs to two different models (e.g., a fast model and a more capable one). Keep parameters like temperature and max tokens consistent. Run each several times to average out variance.

Step 3: Capture metered usage per attempt

For each request, log:

• Model name
• Input and output tokens
• Whether the output met success criteria
• If not, why (e.g., truncated, missing fields)

Your aggregator's metered log should provide token counts; you can add success flags yourself.

Step 4: Account for retries and truncations

If an attempt fails, you may retry with the same or a different model. Each retry adds cost. Also note truncations: a model that hits max tokens without completing the task is a failed attempt even if it returned text.

Example: Model A costs less per token but fails 2 out of 5 times, requiring retries. Model B costs more per token but succeeds 5 out of 5. The effective cost per completed task may favor B.

Step 5: Compute effective cost per completed task

For each model, sum the USDC cost of all attempts (including retries) and divide by the number of successfully completed tasks.

Effective cost = total USDC spent on all attempts / number of completed tasks

Remember your pricing: you pay official price × 1.3. So convert token usage to cost using that multiplier.

Step 6: Compare and decide

Compare the effective costs. Also consider non-cost factors like latency and output quality, but let cost per completed task guide model selection for high-volume work.

Practical tips

• Use a consistent success checker (script or manual) across models.
• Run enough trials to reduce noise.
• Log everything: model, tokens, success, retries.
• Re-evaluate periodically as models update.

Note on contributor credits

If you contribute keys, you are credited official price × 1.1 (premium × 1.2) in USDC. This does not change your cost as a user, but it affects the economics if you both use and contribute.

By focusing on cost per completed task, you make decisions based on real outcomes, not sticker prices.