Migrating a LangChain Agent to the Aggregator: Which Tool-Calling and Streaming Features Survive the Proxy

What the proxy changes—and what it doesn't

The aggregator proxy forwards your requests to the underlying model provider (such as Anthropic, OpenAI, DeepSeek, Qwen, GLM, or Kimi) using its native API format. It does not rewrite LangChain abstractions; it simply passes through the request body and response stream. So most LangChain features that depend on the provider's API—like tool-calling and streaming—work the same way they do when calling the provider directly, provided the model supports them.

The main differences are operational: you use one API key for many models, and you pay the official price × 1.3 for usage. No KYC is required; you top up with USDC on Base. When you contribute keys, you're credited official price × 1.1 (or × 1.2 for premium) in USDC. These economics don't affect LangChain's behavior, but they're worth keeping in mind when budgeting.

Tool-calling: largely preserved

Tool-calling (function calling) in LangChain relies on the model's native support and the structure of the tools you define. The proxy passes your tool definitions and the model's tool-call responses without modification.

• Standard tool-calling works as expected for models that support it natively. LangChain's bind_tools or equivalent methods should continue to function.
• Model-specific quirks (e.g., how a model formats tool calls, parallel tool calls, or forced tool use) are preserved because the proxy doesn't alter the payload. If a model supports parallel tool calls, you'll see them; if it doesn't, you won't.
• Error handling for invalid tool schemas or unsupported features comes from the provider, so you may see the same errors you'd get when calling the provider directly.

One caveat: if you rely on LangChain's built-in retry or fallback logic, verify that the proxy's error responses match what LangChain expects. In most cases they do, because the proxy returns provider errors verbatim.

Streaming: intact, with standard behavior

Streaming responses (e.g., streaming=True or astream) work through the proxy. The proxy relays the provider's server-sent events (SSE) stream without buffering the entire response, so you get tokens as they're generated.

• Token-by-token streaming is preserved, including for tool-call deltas that arrive incrementally.
• Streaming with tool-calling is supported as long as the underlying model supports it. LangChain's stream handling will assemble tool calls from the deltas.
• Timeouts and keep-alives are handled by the proxy; if you see connection issues, check your client's timeout settings and the proxy's documentation for any limits.

Note that some models may not support streaming with tool-calling simultaneously; that's a model limitation, not a proxy limitation.

Testing before you migrate

Before switching a production agent, run a quick compatibility test:

1. Tool-calling round-trip: Define a simple tool (e.g., a weather lookup stub) and confirm the model invokes it correctly through the proxy.
2. Streaming output: Stream a response and verify tokens arrive incrementally.
3. Combined scenario: Test a streaming tool-calling interaction if your agent uses both.
4. Error cases: Send a malformed tool schema and confirm the error propagates as expected.

Use the same LangChain version and tool definitions you plan to use in production.

Configuration tips

• Base URL: Point your LangChain model to the aggregator's endpoint. Most LangChain integrations accept a base_url or similar parameter.
• API key: Use your aggregator key; it works across all supported models.
• Model names: Use the exact model identifiers provided by the aggregator.
• Fallbacks: If you use LangChain's fallback mechanisms, ensure the proxy's error codes and messages are compatible.

Limitations to watch

• Model support varies: Not all models support tool-calling or streaming. Check the aggregator's model documentation for capabilities.
• Proxy-specific features: The proxy may add headers or metadata; these don't affect LangChain's parsing.
• Rate limits: The proxy may impose its own rate limits; these are separate from provider limits. Refer to the aggregator's docs.

In summary, LangChain's tool-calling and streaming features survive the proxy because the proxy is a transparent pass-through. The main work is verifying that your chosen models support the features you need and that your client configuration points to the aggregator correctly.