Google AI
Gemini's OpenAI-compatible interface for capability-gap testing.
Protocol Compatibility Guide
OpenAI-Compatible API: Verify the Semantics
Test an OpenAI-compatible API against the response, tool-call, streaming, and error semantics your application needs.
Published: March 15, 2026 · By the QVeris Team
On this page
Summary
Compatibility has layers
Transport, authentication, endpoint paths, JSON shape, stream events, errors, usage accounting and feature behavior can each differ.
Test features, not slogans
Create contract tests for roles, tools, structured output, streaming, usage, errors, reasoning controls, images, files and audio.
Preserve provider escape hatches
A normalized request should allow vendor-specific parameters and headers where critical capability would otherwise be lost.
Migrate by workload
A chat demo is not evidence for agents, extraction, long streams, multimodal input or batch jobs. Canary each workload separately.
Provider types offer different compatibility guarantees
The original OpenAI API is the reference surface, while cloud-hosted deployments, model marketplaces, hosted inference platforms, managed gateways and self-hosted proxies implement selected parts of that surface. Some expose Chat Completions only; others also implement Responses-like, embeddings, images, audio or batches.
Compatibility can be native, translated or emulated. Native endpoints still vary by model. Translation can normalize common fields but reshape errors or streams. Emulation may accept a parameter without preserving its behavior. The only reliable answer is a versioned conformance suite against exact models and endpoints.
Common OpenAI-compatible provider patterns
| Provider pattern | Best fit | Verify before choosing |
|---|---|---|
| OpenAI reference API | Applications targeting the original endpoint and newest first-party semantics. | Pin endpoint, model and API behavior; even first-party models differ in tools, reasoning, modalities and limits. |
| Azure OpenAI | Enterprises needing Azure identity, networking, regional deployment, governance and commercial alignment. | Validate API versions, deployment names, regional model availability, limits, content filters and error semantics. |
| OpenRouter | Broad hosted model/provider selection behind one OpenAI-style interface with routing controls. | Verify each model/provider endpoint, supported parameters, provenance, privacy, usage, error and billing behavior. |
| LiteLLM Proxy | Self-hosted translation across many providers with gateway policy, keys, budgets and routing. | Own adapter updates and test pass-through, streaming, tools, structured output, errors, usage and provider extensions. |
| Bifrost Gateway | Self-hosted unified API and Go stack with provider routing, failover and operational control. | Test exact protocol endpoints, model features, stream events, governance rules, cluster behavior and upgrade cadence. |
| Hosted inference platforms | Direct hosted access to open or proprietary models, often with an OpenAI-compatible chat path. | Check whether compatibility covers only chat or also tools, structured output, multimodal, embeddings, files, batches and errors. |
| Managed AI gateways | One endpoint adding routing, budgets, observability, fallbacks and provider credentials. | Distinguish protocol compatibility from product SDK compatibility; inspect transformations and provider-specific escapes. |
Eight fidelity checks catch most migration failures
Roles and content
System, developer, user, assistant and tool messages; text, image, file and audio parts; ordering and truncation.
Tool calling
Tool schema limits, choice controls, parallel calls, argument JSON, call IDs, streaming deltas and tool-result correlation.
Structured output
JSON mode versus schema enforcement, unsupported keywords, refusal shape, partial output and validation errors.
Streaming
Event types, ordering, role deltas, finish reasons, usage timing, keepalives, disconnects and mid-stream errors.
Usage and billing
Prompt, completion, reasoning, cache and tool tokens; native versus normalized counts; request IDs and price units.
Errors and retries
HTTP codes, error types, retryability, rate-limit headers, timeout, safety rejection, partial charge and idempotency.
Create a provider conformance suite
- Capture golden requests and expected invariants for every production endpoint, model, feature and failure path.
- Run the same payload through direct and compatible endpoints; diff JSON fields, stream events, tool calls, errors and usage.
- Test unknown parameters, provider extensions, large schemas, long context, invalid tools, disconnects and rate limits.
- Version results by date, API version, model revision, provider endpoint, region and gateway release.
- Fail the deployment when a required invariant regresses; do not silently drop fields or downgrade behavior.
Use a canonical contract with explicit adapters
Define the smallest canonical request and response your applications truly need. Keep provider adapters typed and versioned. Record transformations, unsupported fields and default substitutions. Expose an escape hatch for native parameters, but isolate it behind capability flags. Normalize identifiers and evidence fields without pretending that every model shares one semantic feature set.
Production rule: accepting the same JSON shape is interface compatibility; preserving behavior under success and failure is production compatibility.
Protocol compatibility is not capability discovery
OpenAI-compatible APIs standardize access to model inference. QVeris operates at a different boundary: an agent discovers, inspects and calls external APIs, tools, services and live data. A QVeris capability contract may use any underlying protocol while exposing task-relevant schema and evidence.
Run a compatibility matrix, not a single hello-world
An endpoint can accept /chat/completions and still diverge on tool arguments, structured output, streaming frames, usage accounting, errors, or cancellation. Use a small repeatable suite and store the exact endpoint version with every result.
Minimal HTTP probe
curl --fail-with-body --max-time 20 "$LLM_BASE_URL/chat/completions" -H "Authorization: Bearer $LLM_API_KEY" -H "Content-Type: application/json" -H "X-Trace-ID: compat-$(date +%s)" -d '{
"model": "'"$LLM_MODEL_ID"'",
"messages": [{"role":"user","content":"Reply COMPAT_OK"}],
"stream": false,
"max_completion_tokens": 16
}'- Add separate fixtures for tools, JSON schema, streaming, long context, and invalid input.
- Compare status, error body, request ID, exact model, finish reason, and usage against the contract.
- Fail promotion when a required field disappears, even if text generation still works.
Verified implementation reference: OpenAI Chat Completions reference.
Continue with related implementation guides
Use these adjacent guides to compare the next architectural decision, implementation boundary, or operating control without starting the research again.
FAQ
What does OpenAI-compatible mean?
Usually that a service accepts one or more OpenAI-style endpoints or SDK patterns. The supported feature and behavior subset must be documented and tested.
Can we change only the base URL?
Sometimes for simple chat. Production migrations also need model IDs, keys, parameters, tools, streams, errors, usage and retry behavior validated.
Which compatible provider is best?
Choose by required models, regions, features, data terms, routing, reliability, support and tested fidelity—not catalog size alone.
Should we keep direct provider clients?
Keep them for critical native features, incident isolation and rollback until the compatible layer proves full workload fidelity.