Unified API for LLMs
One Interface Without Pretending Models Match
统一 LLM API：统一接口，但不要假设模型完全相同
A unified API gives applications one base URL, authentication contract and request envelope across multiple models or providers. It reduces integration work while adapters, capability flags and native escape hatches preserve real differences.
统一 API 为应用提供跨多个模型或供应商的单一基础地址（Base URL）、认证契约与请求封装。它减少集成工作，同时用适配器、能力标记与原生出口保留真实差异。
TL;DR
Authentication, model catalog, routing, fallback, errors, usage, billing tags and trace fields can share one contract.
Tools, structured output, reasoning, context, files, embeddings, vision, audio and streams vary by model and endpoint.
Accepting the same JSON shape does not guarantee the same semantics, errors, events, usage or quality.
Inventory workloads, define invariants, compare native and unified paths, dual-run, canary and keep rollback ready.
认证、模型目录、路由、回退、错误、用量、计费标签与调用链字段可共享一个契约。
工具、结构化输出、推理、上下文、文件、Embedding、视觉、音频与流会因模型和端点而异。
接受相同 JSON 形状不保证相同语义、错误、事件、用量或质量。
盘点负载、定义不变量、比较原生与统一路径、双轨运行、灰度并保持回滚就绪。
What a unified LLM API normalizes 统一 LLM API 标准化什么
A unified layer accepts a canonical request, validates capability flags, maps the model alias to eligible endpoints, translates provider-specific fields, routes and falls back, normalizes errors and usage, and returns a stable response envelope. Native IDs remain attached for evidence.
统一层接受规范请求、验证能力标记、把模型别名映射到合格端点、转换供应商特定字段、路由与回退、标准化错误和用量，并返回稳定响应封装。原生 ID 应继续附加供证据使用。
The layer may be a hosted aggregator, managed gateway, self-hosted proxy or SDK. Product shape determines where keys, data, telemetry, billing and operations live. Interface design alone does not answer the trust and ownership questions.
该层可以是托管聚合器、托管网关、自托管代理或 SDK。产品形态决定密钥、数据、遥测、计费与运维位于何处；接口设计本身不能回答信任与责任问题。
Unified API building blocks 统一 API 构建块
| Building block 构建块 | Best fit 最适合 | Verify before choosing 选择前验证 |
|---|---|---|
| One base URL and auth 单一基础地址（Base URL）与认证 | Applications use one service identity and model access boundary. 应用使用一个服务身份与模型访问边界。 | Define tenant isolation, provider keys, BYOK, rotation, least privilege and direct rollback. 定义租户隔离、供应商密钥、BYOK、轮换、最小权限与直连回滚。 |
| Model catalog and flags 模型目录与 Flag | Stable aliases map to versioned capabilities and eligible endpoints. 稳定别名映射到版本化能力与合格端点。 | Source, verify and refresh context, modalities, parameters, region, policy, price and lifecycle. 为上下文、模态、参数、区域、策略、价格与生命周期标注来源、验证并刷新。 |
| Provider adapters 供应商适配器 | Translate canonical requests, responses, streams, errors and usage. 转换规范请求、响应、流、错误与用量。 | Contract-test every required feature and preserve provider-specific escape hatches. 对每个所需功能做契约测试，并保留供应商特定出口。 |
| Routing and fallback 路由与回退 | Select among eligible models or providers and recover from bounded failures. 在合格模型或供应商间选择，并从有边界故障恢复。 | Record candidate set, decision, policy, attempts, substitutions, charges and final error. 记录候选集合、决策、策略、尝试、替换、费用与最终错误。 |
| Usage and evidence 用量与证据 | Normalize operational fields for observability, billing and governance. 为可观测、计费与治理标准化运营字段。 | Store native units and IDs beside normalized data to reconcile providers and incidents. 把原生单位与 ID 同标准化数据并存，以核对供应商与事故。 |
Normalize transport, preserve semantics 统一传输，保留语义
Messages, common tools, model alias, timeout, trace, tenant, budget and required capability flags.
Provider or modality-specific fields isolated behind documented namespaces and feature detection.
Canonical request ID plus native provider IDs, transformations, route, attempts, stream, errors, usage and cost.
Reject missing capabilities before execution; never silently ignore required fields or downgrade semantics.
Message、常用工具、模型别名、超时、调用链、租户、预算与所需能力标记。
供应商或模态特定字段隔离在有文档的命名空间与功能检测后。
规范请求 ID 加原生供应商 ID、转换、路由、尝试、流、错误、用量与成本。
执行前拒绝缺失能力；不得静默忽略必需字段或降级语义。
Migrate to a unified API safely 安全迁移到统一 API
- Inventory models, endpoints, SDKs, features, provider extensions, keys, data terms, errors and billing.
- Build golden contract tests for every production workload and native baseline.
- Dual-run unified and native paths; diff outputs, events, errors, usage, latency and cost.
- Canary by workload, preserve direct rollback and block deployment on required invariant regressions.
- 盘点模型、端点、SDK、功能、供应商扩展、密钥、数据条款、错误与计费。
- 为每个生产负载与原生基线构建 Golden 契约测试。
- 双轨运行统一与原生路径，对比输出、事件、错误、用量、延迟与成本。
- 按工作负载灰度，保留直连回滚，并在必需不变量回归时阻止部署。
Use a canonical envelope with typed adapters 使用规范封装与类型化适配器
The canonical envelope contains identity, trace, workload alias, messages, required capabilities, timeout, budget and policy. A catalog resolves eligible endpoints before translation. Typed adapters map only supported semantics. The response includes normalized output plus route and native evidence. Escape hatches are explicit, scoped and covered by tests.
规范封装包含身份、调用链、工作负载别名、Message、所需能力、超时、预算与策略。目录在转换前解析合格端点；类型化适配器只映射受支持语义；响应包含标准化输出以及路由和原生证据；出口应显式、受限并被测试覆盖。
Production rule: a unified API should make common operations portable without erasing the differences that determine correctness.
生产规则：统一 API 应让通用运营可移植，但不能抹去决定正确性的差异。
A unified LLM API is still an inference interface 统一 LLM API 仍是推理接口
The unified API gives an agent consistent access to models. QVeris gives that agent consistent discovery and execution of external APIs, tools, services and live data. Stack them as inference interface plus capability interface, with shared workflow trace context.
统一 API 让智能体一致访问模型；QVeris 让智能体一致发现和执行外部 API、工具、服务与实时数据。把两者作为推理接口与能力接口叠加，并共享工作流调用链上下文。
Define the Production Contract for a unified API for LLMs定义统一 LLM API的生产契约
For Unified API for LLMs, protocol similarity lowers integration effort, but it does not guarantee behavioral parity. Put a versioned application contract between product code and the provider path so change remains testable and reversible.
针对“统一 LLM API”，协议相似可以降低集成工作量，却不能保证行为等价。应在产品代码与供应商路径之间建立版本化应用契约，让变更保持可测试、可回滚。
Document canonical request and response envelopes, provider-native extensions, model identity, streaming events, tools, usage accounting, errors, versioning, and escape hatches. Mark each item as required, optional, provider-native, or unsupported, and assign an owner for any transformation that changes its meaning.
记录规范请求与响应封装、供应商原生扩展、模型身份、流式事件、工具、用量计量、错误、版本化和逃生通道。把每一项标记为必需、可选、供应商原生或不支持，并为任何改变语义的转换明确负责人。
To validate Unified API for LLMs, create fixtures for short and long prompts, Unicode, streaming, JSON schema, parallel tools, refusals, cancellation, malformed input, and rate limits. Check required fields and event order instead of accepting one plausible text answer.
验证“统一 LLM API”时，为短与长提示词、Unicode、流式、JSON Schema、并行工具、拒绝、取消、畸形输入和限流建立 Fixture，检查必需字段与事件顺序，而不是接受一个看似合理的文本答案。
When operating Unified API for LLMs, trace internal request ID, resolved model and provider, model version, transformations, retries, latency, token classes, cost source, policy result, and output validation. Redact secrets without deleting the context needed to reproduce failure.
运营“统一 LLM API”时，追踪内部请求 ID、解析后的模型与供应商、模型版本、转换、重试、延迟、Token 类别、成本来源、策略结果和输出校验，在脱敏密钥的同时保留复现失败所需上下文。
Before rolling out Unified API for LLMs, shadow representative traffic, classify semantic differences, canary by reversible workload, monitor task completion and tail behavior, and keep the previous route available until rollback and return-to-primary have been rehearsed.
上线“统一 LLM API”前，运行代表性影子流量，分类语义差异，按可逆工作负载进行金丝雀发布，监控任务完成与长尾行为，并在演练回滚和恢复主路径之前保留原有路由。
FAQ
One client contract that accesses multiple models or providers through shared authentication, requests, routing and operational evidence.
No. It simplifies transport and operations; capabilities, behavior, quality, policy and price remain model- and endpoint-specific.
For simple compatible calls, sometimes. Production workloads still require conformance, dual-run, canary and rollback.
通过共享认证、请求、路由与运营证据访问多个模型或供应商的单一客户端契约。
不会。它简化传输与运营；能力、行为、质量、策略与价格仍因模型和端点而异。
简单兼容调用有时可以；生产负载仍需一致性测试、双轨、灰度与回滚。