09 — Provider APIs and the OpenAI-Compatible Standard

The API layer between my code and the model used to feel like a different language per provider — until I noticed most of them speak the same dialect. The load-bearing fact: each provider has its own native API shape, but OpenAI's request format became the de facto standard, so any provider that mimics it lets me target many models with one codebase. [1] That single fact is why "switching providers" can be a config change instead of a rewrite, and it's the idea behind routers, fallbacks, and most of the inference SDK ecosystem.

The framing that finally landed is two layers. The bottom layer is the native API each vendor ships — OpenAI's Responses API, Anthropic's Messages API, Google's Gemini API — each shaped around that vendor's view of how a conversation should be modeled. The top layer is the compatibility layer: because so much code was written against OpenAI's format, providers now offer OpenAI-compatible endpoints, and SDKs and routers speak that one dialect. When I write application code, I'm almost always talking to the top layer, and I drop to a native API only when I need a feature the standard doesn't cover.

The native APIs: each vendor's view of a conversation

The big three each ship a native API shaped by their model's design.

OpenAI's Responses API is the newer entry, intended to eventually replace the older Chat Completions and Assistants APIs for new projects [2]. It provides a structured way to send prompts with parameters (temperature, max length, tools) and receive generated text or other outputs. The Responses API is OpenAI's native shape — but its _Chat Completions_ shape is the one the rest of the industry cloned.

Claude's Messages API models interaction as a series of messages — a conversation — and supports text, images, and structured data in those messages [3]. It's shaped around Anthropic's view of safe, structured interaction, and it has features (like first-class prompt caching and extended thinking) that the OpenAI-shaped standard doesn't always expose.

Google's Gemini API gives programmatic access to Gemini's multimodal family — text, images, audio, video — through Google's generateContent-style calls [4]. It's shaped around multimodal content parts and Google's auth conventions.

The point: these APIs are not interchangeable at the native level. The request body, the auth, the parameter names, the response shape — all differ. Writing directly to a native API couples my code to that one vendor.

The OpenAI-compatible standard

Here's where the picture simplifies. Because OpenAI's Chat Completions format was first and had the most code written against it, that format became the _de facto_ standard. OpenAI-compatible APIs are interfaces that mimic OpenAI's request and response shape, so code written for OpenAI works against them with minimal changes — usually just swapping the base URL and API key [1].

The payoff is enormous. Providers that offer an OpenAI-compatible endpoint let me:

• Swap models without rewriting code — change the model name in the request.
• Mitigate vendor lock-in — my application isn't tied to one provider's native API.
• Build fallback mechanisms — if one provider is down, route to another that speaks the same dialect.

Most major providers and most local runners (Ollama, LM Studio) now expose an OpenAI-compatible endpoint, which is why a single OpenAI SDK can often target all of them. The Gemini API even ships an explicit OpenAI compatibility layer for exactly this reason [1].

Inference SDKs: the client libraries

On top of the API layer sit inference SDKs — client libraries that handle the HTTP, retries, streaming, and typed responses. The Hugging Face InferenceClient, for example, lets me call many models hosted on the Hub through one consistent SDK, with both synchronous and asynchronous operations [5]. The OpenAI SDK, used against any OpenAI-compatible endpoint, does the same for the broader provider market.

The SDK is where ergonomics live. Raw HTTP works, but typed responses, automatic retries, and first-class streaming turn "I can call the model" into "I can build a feature on top of it without losing weekends to plumbing." When I pick a provider or a router, the quality of its SDK is now a real part of the decision, not an afterthought.

When to use the native API anyway

The compatibility layer covers most cases, but I drop to a native API when I need a feature the standard doesn't expose. Examples that have come up:

• Prompt caching as a first-class concept (Claude's Messages API handles this explicitly).
• Extended thinking / reasoning modes that don't map onto the standard parameters.
• Provider-specific multimodal inputs that the OpenAI-shaped format represents awkwardly.
• Auth patterns tied to a vendor's ecosystem (Google service accounts, for instance).

The discipline is to start against the compatibility layer — it's portable and good enough most of the time — and reach for the native API only when a concrete feature demands it. Coding to native APIs by default throws away the portability for no reason.

How I use this

My default is to write against the OpenAI SDK pointed at whatever provider I'm using, including local runners like Ollama. That single habit keeps my codebase portable: switching providers, adding a fallback, or moving from cloud to local is a config change, not a refactor. I reach for a native API — Claude's Messages API for prompt caching, Gemini's for a specific multimodal input — only when I've confirmed the feature needs it. And I treat the SDK quality as a real selection criterion, because the difference between a good SDK and a bad one shows up every day I work on the feature. The standard is what makes inference engineering tractable; without it, every provider switch would be a project.

References

[1] BentoML, "OpenAI-compatible API," 2024. [Online]. Available: https://bentoml.com/llm/llm-inference-basics/openai-compatible-api

[2] OpenAI, "Responses API," 2024. [Online]. Available: https://developers.openai.com/api/reference/resources/responses/

[3] Anthropic, "Messages API," 2024. [Online]. Available: https://platform.claude.com/docs/en/api/messages

[4] Google, "Gemini API," 2024. [Online]. Available: https://ai.google.dev/gemini-api/docs

[5] Hugging Face, "Inference Client," 2024. [Online]. Available: https://huggingface.co/docs/huggingface_hub/en/package_reference/inference_client