Migrate From OpenAI API
Safely With Rollback ReadyMigrate OpenAI API
Safely
A safe migration must preserve tools, structured output, streams, errors, usage, latency, and quality under real workloads.
Published: March 21, 2026 · By the QVeris Team
TL;DR
Find endpoints, SDK versions, models, prompts, tools, streams, files and batch jobs.
Compare behavior and constraints by feature, not by marketing label.
Normalize only what the application contract requires and preserve native fields.
Shift one workload gradually while measuring semantic and operational differences.
Migration scope is larger than the endpoint
Catalog every OpenAI API path and feature the application actually uses: Responses or Chat Completions input shape, roles and instructions, structured output, tools, streaming events, images, files, embeddings, Batch, moderation, usage fields, request IDs and error handling. The current OpenAI quickstart uses the Responses API, but a migration plan must describe the endpoint your production workload really depends on rather than assuming all text APIs behave alike.
Classify workloads by business criticality and feature depth. A stateless summarizer may need only text and token accounting, while an agent can depend on tool-call IDs, approval state, stored conversation objects or server-side tools. Also record data-retention and regional requirements because changing the endpoint or provider can change where application state exists and how long it persists.
Freeze representative prompts, tool schemas, sample inputs and expected behaviors in a golden corpus. The new route should be judged against workload acceptance criteria—factual coverage, schema validity, tool selection, safety and latency—rather than exact text equality. Pin the source model snapshot during the comparison so model drift is not mistaken for adapter drift.
Compatibility dimensions
| Dimension | Best fit | Verify before choosing |
|---|---|---|
| Messages and output | Core text workloads with stable input and output contracts. | Roles, instruction precedence, stop behavior and structured output. |
| Tools and streams | Agents or interactive applications. | Tool schema, call IDs, parallelism, chunk ordering and termination. |
| Errors and retries | Production reliability paths. | Codes, retry hints, timeout ownership, idempotency and cancellation. |
| Usage and terms | Cost, audit and compliance owners. | Units, cached usage, tools, retention, regions, support and billing. |
Release gates
Golden tasks meet quality, safety and tool correctness thresholds.
Latency, throughput, cancellation, retries and alerts satisfy SLOs.
Usage can be reconciled to provider evidence and invoices.
Old route, configuration and data dependencies remain ready during canary.
Eight-step migration sequence
Move one workload class at a time. The sequence below separates discovery, compatibility work and traffic change so a failed evaluation does not become a production incident.
- Inventory endpoints, models, SDK versions, stored state and downstream consumers.
- Classify workloads and define behavioral, operational, accounting and compliance gates.
- Build a feature matrix and identify unsupported or differently modeled capabilities.
- Implement a versioned adapter and preserve provider-native request and response evidence.
- Run the deterministic golden evaluation suite against pinned source and target routes.
- Dual-run sampled, privacy-safe traffic and compare semantics, latency, errors and usage accounting.
- Canary by workload and tenant while monitoring explicit promotion and stop thresholds.
- Exercise rollback under load, then cut over only after the rollback route remains proven.
Use an adapter and a versioned route
The application should call a stable internal contract rather than a provider object throughout the codebase. A versioned adapter maps authentication, endpoint paths, input items or messages, tool definitions, structured output, streaming events, errors, request IDs and usage to the target foundation. Keep unsupported fields explicit; silently dropping a parameter creates a migration that appears successful while changing behavior.
Routing policy sits above the adapter and supports shadow execution, canary percentages, tenant allowlists and one-switch rollback without rebuilding the application. Store the route and adapter version with each trace. During dual run, prevent the shadow path from executing real tool side effects or persisting duplicate conversation state.
Production rule: do not cut over until behavioral, operational, accounting, compliance and rollback gates all pass.
Migrate inference without mixing tool access
The migration changes the model API path. QVeris independently governs discovery and execution of external APIs, tools, services and live data. Preserve tool schemas, credential ownership, tenant permissions, idempotency keys and traces across the model migration rather than rebuilding the capability layer at the same time.
Evaluate tool selection and argument generation on both routes, but execute side effects only on the designated primary. If the target model represents tool calls differently, normalize them at the adapter boundary and retain the provider-native payload for diagnosis. This isolates inference differences without weakening approval, audit or duplicate-prevention controls.
Move one reversible configuration boundary
Start by moving only the client construction behind environment-owned configuration. Do not simultaneously rename models, change prompts, enable routing, and alter retries; those changes make regressions impossible to attribute.
from openai import OpenAI
import os
def build_client() -> OpenAI:
return OpenAI(
api_key=os.environ["LLM_API_KEY"],
base_url=os.environ.get("LLM_BASE_URL", "https://api.openai.com/v1"),
timeout=float(os.environ.get("LLM_TIMEOUT_SECONDS", "30")),
max_retries=0,
)
MODEL_ALIASES = {
"support-fast": os.environ["SUPPORT_FAST_MODEL"],
"support-quality": os.environ["SUPPORT_QUALITY_MODEL"],
}
client = build_client()
model = MODEL_ALIASES["support-fast"]- Run golden prompts through old and new endpoints with the same model contract.
- Compare accepted quality, latency, errors, tool behavior, usage, and total cost.
- Canary by workload or tenant and keep the previous URL, key reference, and alias map ready.
Verified implementation reference: OpenAI Python library.
Continue with related implementation guides
Use these adjacent guides to compare the next architectural decision, implementation boundary, or operating control without starting the research again.
FAQ
Not automatically. Test every required feature and failure path.
Usually no. Compare task acceptance, safety, schema and tool behavior.
Until representative traffic covers risk classes and all release gates are stable.