Switch LLM Providers
Treat the Base URL Swap as Step One
切换 LLM 供应商：更换基础地址（Base URL）只是第一步
Provider migration is a behavioral change, not only a transport change. Inventory dependencies, test real workloads, dual-run both paths, canary by risk and keep the old route ready for rollback.
供应商迁移是行为变化，而不仅是传输变化。盘点依赖、测试真实负载、双轨运行两条路径、按风险灰度，并让旧路径随时可回滚。
TL;DR
Find every model, endpoint, SDK, key, prompt, tool, schema, stream, policy and billing dependency.
Compare transport plus tools, output, errors, usage, context, modalities, data and lifecycle.
Replay representative requests through current and target paths and diff accepted behavior.
Keep credentials, routing, dashboards and runbooks ready until evidence proves stability.
找出每个模型、端点、SDK、密钥、提示词、工具、结构定义、流、策略与计费依赖。
比较传输以及工具、输出、错误、用量、上下文、模态、数据与生命周期。
通过当前与目标路径回放代表性请求，并对比可接受行为。
在证据证明稳定前，保留凭证、路由、仪表盘与 Runbook。
Why a base URL swap is not enough 为什么只换基础地址（Base URL）不够
A compatible target may accept the same JSON while differing in model behavior, default parameters, tool calling, structured output, stream events, errors, usage accounting, latency, pricing, policy and data handling. The application can remain syntactically healthy while outcomes regress.
兼容目标可能接受相同 JSON，却在模型行为、默认参数、工具调用、结构化输出、流事件、错误、用量核算、延迟、价格、策略与数据处理上不同。应用可能在语法层面健康，但结果已经回归。
Treat migration as a sequence of reversible gates: inventory, compatibility matrix, golden tests, dual run, diff, canary and rollback-ready cutover. Each gate has workload-specific acceptance criteria and named owners.
把迁移视为一系列可逆 Gate：盘点、兼容矩阵、Golden 测试、双轨、Diff、灰度与回滚就绪切换。每个 Gate 都有工作负载特定验收标准与负责人。
Migration gates and exit criteria 迁移 Gate 与退出标准
| Gate Gate | Best fit 最适合 | Verify before choosing 选择前验证 |
|---|---|---|
| Inventory 盘点 | Complete map of applications, workloads and operational dependencies. 完整映射应用、工作负载与运营依赖。 | No unknown production caller, key, extension, tool, data policy or billing owner. 不存在未知生产调用方、密钥、扩展、工具、数据策略或计费负责人。 |
| Compatibility matrix 兼容矩阵 | Field-by-field and behavior-by-behavior comparison. 逐字段、逐行为比较。 | Every required gap has a mitigation, typed extension or explicit block. 每个必需差距都有缓解、类型化扩展或明确阻断。 |
| Golden tests Golden 测试 | Deterministic workload suites and accepted invariants. 确定性工作负载测试集与可接受不变量。 | Target passes required tools, schemas, streams, errors, safety and output checks. 目标通过必需工具、结构定义、流、错误、安全与输出检查。 |
| Dual run and diff 双轨与 Diff | Current and target execute the same representative traffic. 当前与目标执行相同代表性流量。 | Structural, semantic, usage, latency and cost deltas fit the acceptance envelope. 结构、语义、用量、延迟与成本差异位于验收范围。 |
| Canary and cutover 灰度与切换 | Small risk-bounded traffic moves with live gates. 小规模、有风险边界的流量在实时 Gate 下迁移。 | Quality and SLOs remain stable; rollback switch, evidence and owners are ready. 质量与 SLO 稳定；回滚开关、证据与负责人就绪。 |
Compare behavior, not only schemas 比较行为，而不只比较结构定义
Tools, strict output, reasoning, context, modalities, streaming and stop behavior.
Timeouts, status codes, error bodies, rate-limit hints, retryability and partial streams.
Native usage units, price dimensions, latency distribution, retries and review cost.
Data terms, region, retention, safety, versioning, deprecation and support.
工具、严格输出、推理、上下文、模态、流与停止行为。
超时、状态码、错误体、限流提示、可重试性与部分流。
原生用量单位、价格维度、延迟分布、重试与审核成本。
数据条款、区域、保留、安全、版本、弃用与支持。
Run a reversible migration 执行可逆迁移
- Freeze the current production baseline and collect representative workload samples.
- Contract-test the target and block unsupported required capabilities before execution.
- Dual-run with redacted or approved data and store comparable evidence.
- Canary by workload risk, tenant or traffic slice with automatic stop gates.
- Keep the previous route hot until the stability window and rollback drill pass.
- 冻结当前生产基线并收集代表性工作负载样本。
- 契约测试目标，并在执行前阻断不支持的必需能力。
- 使用已脱敏或批准数据双轨运行，并存储可比证据。
- 按工作负载风险、租户或流量切片灰度，并设置自动停止 Gate。
- 在稳定窗口与回滚演练通过前，让旧路由保持热备用。
Put a migration switch above both providers 在两个供应商之上设置迁移开关
Applications call a stable workload interface. A versioned route policy sends approved traffic to current, target or dual-run paths. Provider adapters preserve explicit differences. A diff pipeline compares normalized and native evidence. Canary gates change policy gradually; the rollback switch restores the prior route without an application redeploy.
应用调用稳定工作负载接口；版本化路由策略把批准流量发送到当前、目标或双轨路径；供应商适配器保留显式差异；Diff Pipeline 比较标准化与原生证据；灰度 Gate 渐进改变策略，回滚开关无需重新部署应用即可恢复旧路由。
Production rule: do not decommission the prior path until rollback has been exercised under production-like load.
生产规则：在生产相似负载下完成回滚演练前，不得下线旧路径。
Keep capability integrations stable during model migration 模型迁移期间保持能力集成稳定
QVeris sits above the model gateway as the complementary interface for external APIs, tools, services and live data. Keep those capability contracts and trace fields stable while the inference provider changes. This isolates model migration from the real-world integrations an agent depends on.
QVeris 位于模型网关之上，作为外部 API、工具、服务与实时数据的互补接口。在推理供应商变化时保持这些能力契约与调用链字段稳定，从而把模型迁移与智能体依赖的真实世界集成隔离。
Switch configuration only after contract evidence passes 只有契约证据通过后才切换配置
Treat a provider switch as a release with an artifact: exact model, endpoint, adapter version, test results, rollout cohort, and rollback target. This separates a reversible route change from an application deployment.
把供应商切换当作一次有 Artifact 的发布：记录准确模型、端点、适配器 Version、测试结果、Rollout Cohort 与 Rollback Target，使可逆路由 Change 与应用部署分离。
change_id: llm-route-2026-07-21-01
workload: support-answer
from: provider-a/model-snapshot-a
to: provider-b/model-snapshot-b
contract_suite: contracts/support-answer-v4.yaml
evidence: artifacts/support-answer-provider-b.json
canary_percent: 5
success_gate:
accepted_quality: ">= baseline"
p95_latency_ms: "<= 8000"
error_rate: "<= 1%"
rollback:
alias_map: config/aliases-v18.yaml
owner: platform-oncall
- Change one dimension at a time: route, model, prompt, or retry policy.
- Use shadow traffic only when data terms permit it and outputs cannot cause side effects.
- Promote gradually and retain evidence for both success and rollback decisions.
- 一次只改变一个维度：路由、模型、提示词或 Retry Policy。
- 只有数据条款允许且输出不会产生副作用时才使用 Shadow Traffic。
- 逐步推广，并保留成功与回滚决策的证据。
Verified implementation reference: OpenTelemetry GenAI conventions.
实施参考已根据官方资料核验：OpenTelemetry GenAI conventions。
Define the Production Contract for switching LLM providers定义切换 LLM 供应商的生产契约
For Switch LLM Provider, protocol similarity lowers integration effort, but it does not guarantee behavioral parity. Put a versioned application contract between product code and the provider path so change remains testable and reversible.
针对“切换 LLM 供应商”，协议相似可以降低集成工作量，却不能保证行为等价。应在产品代码与供应商路径之间建立版本化应用契约，让变更保持可测试、可回滚。
Document model capability, prompt assumptions, safety behavior, context and tokenizer, structured output, tools, latency, quotas, price, region, and support. Mark each item as required, optional, provider-native, or unsupported, and assign an owner for any transformation that changes its meaning.
记录模型能力、提示词假设、安全行为、上下文与 Tokenizer、结构化输出、工具、延迟、配额、价格、区域和支持。把每一项标记为必需、可选、供应商原生或不支持，并为任何改变语义的转换明确负责人。
To validate Switch LLM Provider, create fixtures for short and long prompts, Unicode, streaming, JSON schema, parallel tools, refusals, cancellation, malformed input, and rate limits. Check required fields and event order instead of accepting one plausible text answer.
验证“切换 LLM 供应商”时，为短与长提示词、Unicode、流式、JSON Schema、并行工具、拒绝、取消、畸形输入和限流建立 Fixture，检查必需字段与事件顺序，而不是接受一个看似合理的文本答案。
When operating Switch LLM Provider, trace internal request ID, resolved model and provider, model version, transformations, retries, latency, token classes, cost source, policy result, and output validation. Redact secrets without deleting the context needed to reproduce failure.
运营“切换 LLM 供应商”时，追踪内部请求 ID、解析后的模型与供应商、模型版本、转换、重试、延迟、Token 类别、成本来源、策略结果和输出校验，在脱敏密钥的同时保留复现失败所需上下文。
Before rolling out Switch LLM Provider, shadow representative traffic, classify semantic differences, canary by reversible workload, monitor task completion and tail behavior, and keep the previous route available until rollback and return-to-primary have been rehearsed.
上线“切换 LLM 供应商”前，运行代表性影子流量，分类语义差异，按可逆工作负载进行金丝雀发布，监控任务完成与长尾行为，并在演练回滚和恢复主路径之前保留原有路由。
FAQ
Only for very simple workloads after tests prove the target preserves every required behavior.
Representative production workloads, especially tools, strict schemas, streams and failure paths.
After the stability window, reconciliation, incident review and a successful rollback drill.
仅适用于测试证明目标保留全部必需行为的非常简单负载。
代表性生产负载，尤其是工具、严格结构定义、流与故障路径。
在稳定窗口、核对、事故复盘与成功回滚演练之后。