The OpenAI-Compatible /chat/completions Endpoint Explained: What Stays the Same and What Changes on the Aggregator

Why the request shape matters more than the SDK

Most client libraries for OpenAI-compatible APIs are thin wrappers around one HTTP POST to /chat/completions. If you understand the JSON body, you understand the integration. The aggregator's goal is to keep that body as close to the standard as possible, so the interesting question is not "what do I send" but "which fields still mean what I expect".

This page walks the request top to bottom and flags the fields where behavior can differ from calling a vendor directly.

The endpoint and authentication

At the transport level, nothing unusual happens:

• POST {baseurl}/chat/completions with Content-Type: application/json and Authorization: Bearer <yourkey>.
• The response is a JSON object in the standard chat completion envelope (id, object, created, model, choices, usage).
• Streaming uses text/event-stream with data: chunks and a terminating data: [DONE].

The meaningful difference is in the key itself. One aggregator key authenticates requests that may be routed to different upstream vendors. That means the key is a routing credential, not a vendor credential, and the same header works regardless of which model you name. Any code that inspects the key prefix to infer a vendor will be wrong here.

model: the field that changes the most

The syntax is identical — a string. The semantics are not.

• On a direct vendor API, accepted model strings are that vendor's catalog.
• On the aggregator, the string is a routing instruction. Different vendors' model families are reachable with the same key, so the model identifier is the only thing that decides where the request lands.
• Model names are case-sensitive and versioned; treat them as opaque identifiers from the aggregator's own model list rather than strings you construct or guess.
• Because vendors update and retire versions on their own schedules, the set of valid strings is not static. Cache the list, but be prepared to refresh it.

Practical consequence: do not hardcode a model string in more than one place. A single config value you can swap is the difference between a one-line change and a code review.

messages: structurally unchanged, semantically vendor-dependent

The array of role/content objects keeps its standard shape: system, user, assistant, and tool-related roles where supported. What varies is how faithfully each upstream model honors them.

• System prompt handling differs between model families. Some weight it heavily; others follow it loosely.
• Multi-turn history, image parts, and tool-call blocks are only accepted by models that support them. The aggregator passes the structure through; it does not emulate capabilities a model lacks.
• Prompt caching behavior, if a vendor offers it, is vendor-specific and not something the request shape guarantees.

So the field is portable, but the behavior it produces is not. Test prompts against each model you intend to use rather than assuming parity.

Sampling parameters: temperature, topp, maxtokens

These keep their names and general meaning, with two caveats worth knowing before you tune them.

• Ranges and defaults are upstream. A value that is valid for one model family may be rejected or clamped by another. If you build a shared config, keep per-model overrides rather than one global setting.
• Token accounting is per model. max_tokens is measured in the target model's tokenizer, not a universal unit. The same number means different lengths across families.

Treat sampling parameters as portable in syntax and non-portable in effect.

stream and stream_options

Setting stream: true behaves as expected: incremental deltas, a final chunk, and a [DONE] sentinel. Two practical notes:

• Optional stream options such as usage reporting depend on whether the upstream supports them. Do not assume every model returns usage on a streamed response; have a fallback path that estimates or omits it.
• Incremental tool-call arguments can arrive in fragments. Your parser must accumulate them rather than assume each chunk is complete JSON.

Tool calling and structured output

Tool definitions (tools, tool_choice) and JSON-mode style parameters follow the standard shape, but support is a per-model property, not a per-endpoint one.

• A model that does not support tools will not gain that ability by being called through the aggregator.
• Schema strictness and the exact JSON emitted for tool_calls vary by vendor. Validate arguments on your side before executing anything.
• If your application depends on strict structured output, pin the model and verify behavior after any model change rather than relying on the parameter alone.

Fields that are vendor extensions

Some request fields exist only on one vendor's API — reasoning controls, vendor-specific caching hints, safety toggles, or custom metadata. These are the riskiest part of a cross-vendor request body.

• Sending a vendor extension to a model that does not support it may be ignored, may be rejected, or may be passed through with no effect.
• Prefer a minimal, standard body and add extensions only per model, guarded in code.
• If a field is silently ignored, your application may appear to work while the behavior you configured never takes effect. Validate rather than assume.

usage in the response

The usage object is present in the standard place, and it is the basis for billing. Two things to internalize:

• Tokens are counted by the upstream model's tokenizer, so the numbers are model-specific.
• On the aggregator, the price you pay is the official model price multiplied by a fixed factor, and the key that contributed the funding is credited back a portion of the official price. The practical point is that cost is a function of model choice and token count — not of which endpoint path you call.

Do not compute costs from a hardcoded per-token constant. Derive them from the usage you receive and the current pricing shown by the aggregator.

Errors and retries

Status codes follow HTTP conventions (4xx for client problems, 5xx for upstream or transient issues), and error bodies are JSON. Because a single endpoint fronts many upstreams, the same status code can originate from different layers.

• Retry 5xx and timeouts with exponential backoff and jitter.
• Do not blindly retry 4xx; a malformed request or an unsupported parameter for that model will fail identically.
• Include the request id from the response when reporting a problem — it is the fastest way to trace a routing issue.
• Idempotency is not automatic on chat completions. If you add your own retry layer, avoid duplicating expensive non-streamed calls without a guard.

A minimal, portable request body

If you want one body that works across the widest range of models, keep it small:

• model from config, never hardcoded inline.
• messages with plain text content.
• temperature and max_tokens set per model, not globally.
• stream chosen by call site.

Everything else — tools, structured output, vendor extensions — should be added conditionally for models you have tested.

What to trust before you ship

The request shape is trustworthy at the syntax level: endpoint path, auth header, message array, streaming envelope, and usage object all behave as the standard describes. What is not uniform is everything downstream of model — capability support, parameter ranges, tokenizer accounting, and extension handling. Write your client so that model is a single injected value, keep the body minimal by default, and validate capability-dependent fields per model rather than per endpoint.