Asking for Strict JSON Output Across Six Backends on One Key: Where Schema Enforcement Loosens

The setup: one schema, many backends

A single API key that fronts several model families sounds like a uniform interface. It is uniform at the transport layer, not at the schema layer. JSON-mode and structured-output features are implemented per provider and per model generation, so the same request can come back shaped slightly differently depending on which backend actually served it.

This guide describes how to probe that for yourself and what kinds of drift to expect, so you know when your own validation must catch what the model did not.

Define a schema that stresses the weak spots

Pick a small schema that exercises the features most likely to break:

• A required string, an enum, and a number.
• A nested object with its own required fields.
• An array of objects with a minimum sensible length.
• A boolean and a nullable field.

Keep it small enough that a failure is obvious. A schema with two fields rarely exposes drift.

Send the same prompt to every backend

Hold the prompt constant and vary only the model selector. Include the schema in the request wherever the backend supports a structured-output parameter, and fall back to an explicit instruction ("Return only JSON matching this schema; no prose, no code fences") where it does not.

Record for each call:

• Whether the body parsed as JSON on the first attempt.
• Whether it matched the schema exactly.
• The specific drift category, if any.

Common drift categories

Across backends, the same handful of failures recur. Knowing the categories makes them fast to spot.

Wrapping prose and code fences

Some outputs arrive as a JSON object surrounded by an explanatory sentence, or inside triple-backtick fences. This is valid text but not valid payload. It is the most common failure when a backend lacks a true JSON mode and is being steered by prompt instruction alone.

Missing optional-looking required fields

A model may omit a field it considers inferable, even when the schema marks it required. Nested objects are especially prone to losing a leaf field.

Type coercion

Numbers returned as strings, booleans returned as the strings "true"/"false", or integers returned as floats. Schema enforcement at the provider layer constrains shape far more reliably than it constrains scalar types.

Array shape surprises

A single object returned where an array of one was expected, or an empty array returned for a required list. Some backends will happily return {} or [] rather than failing the request.

Enum drift

A value outside the enum, sometimes a close synonym of an allowed value. Strict enum enforcement varies; treat enums as suggestions unless your validator enforces them.

Extra fields

Additional keys not in the schema. Usually harmless if you validate by whitelist, but a problem if you forward the payload somewhere that rejects unknown fields.

Where enforcement is strongest and weakest

As a rule of thumb:

• Native structured output (a provider feature that takes a schema and constrains generation) tends to honor required fields and enums most faithfully.
• JSON mode (guaranteeing parseable JSON without a schema) reliably gives you a parseable object but says little about fields and types.
• Prompt-only instruction gives you the most drift: prose, fences, and missing fields all appear here.

Because the aggregator routes to different providers, the mode actually in effect depends on the model you select, not just on the flag you set. Assume the weakest mode for any backend whose structured-output support you have not verified.

Treat validation as non-optional

Since enforcement loosens unpredictably across backends, the client must be the source of truth. A robust pipeline:

• Strips code fences and extracts the first balanced JSON object before parsing.
• Validates against the schema and rejects on failure rather than coercing silently.
• Applies explicit type coercion only where you have decided it is safe.
• Logs which model served the call alongside the drift category, so patterns become visible.

Build a drift catalogue per model

Before wiring a model into production, run your schema a handful of times and note which drift categories appear. Keep a short internal table: model, mode used, observed drift. This turns an open-ended worry into a known list you can either validate around or route away from.

When to add a repair pass

If a backend drifts often on a particular schema, a cheap second attempt (re-prompt with the validator's error message) is usually enough. Cap retries, and on final failure fail loudly rather than shipping malformed data downstream. The cost of a retry is small compared with the cost of a bad payload; on this platform you pay the official price times 1.3 in USDC, so budget retries explicitly rather than assuming a single call.

Key takeaways

• One key gives you one transport, not one schema contract.
• Probe each backend with a schema that stresses enums, nesting, and scalar types.
• Expect prose wrapping, missing fields, and type coercion as the usual suspects.
• Validate and parse defensively on the client; never trust the backend to enforce your schema.
• Keep a per-model drift catalogue and a capped repair pass.