Unified API for LLMs
One Interface Without Pretending Models Match统一 LLM API：统一接口，但不要假设模型完全相同
A unified API gives applications one base URL, authentication contract and request envelope across multiple models or providers. It reduces integration work while adapters, capability flags and native escape hatches preserve real differences.
统一 API 为应用提供跨多个模型或供应商的单一基础地址（Base URL）、认证契约与请求封装。它减少集成工作，同时用适配器、能力标记与原生出口保留真实差异。
TL;DR
Authentication, model catalog, routing, fallback, errors, usage, billing tags and trace fields can share one contract.
Tools, structured output, reasoning, context, files, embeddings, vision, audio and streams vary by model and endpoint.
Accepting the same JSON shape does not guarantee the same semantics, errors, events, usage or quality.
Inventory workloads, define invariants, compare native and unified paths, dual-run, canary and keep rollback ready.
认证、模型目录、路由、回退、错误、用量、计费标签与调用链字段可共享一个契约。
工具、结构化输出、推理、上下文、文件、Embedding、视觉、音频与流会因模型和端点而异。
接受相同 JSON 形状不保证相同语义、错误、事件、用量或质量。
盘点负载、定义不变量、比较原生与统一路径、双轨运行、灰度并保持回滚就绪。
What a unified LLM API normalizes统一 LLM API 标准化什么
A unified layer accepts a canonical request, validates capability flags, maps the model alias to eligible endpoints, translates provider-specific fields, routes and falls back, normalizes errors and usage, and returns a stable response envelope. Native IDs remain attached for evidence.
统一层接受规范请求、验证能力标记、把模型别名映射到合格端点、转换 Provider 特定字段、路由与回退、标准化错误和用量，并返回稳定响应封装。原生 ID 应继续附加，作为核验依据。
The portable core should be deliberately small: workload alias, messages or input items, common tool schema, required output contract, timeout, tenant, trace, budget, and capability requirements. Everything else—reasoning controls, cache behavior, safety settings, media details, built-in tools, stateful conversations, and provider-specific sampling—should be represented as typed extensions or rejected explicitly when unsupported.
可移植核心应刻意保持精简，包括工作负载别名、Message 或 Input Item、通用工具 Schema、必需输出契约、超时、租户、Trace、预算与能力要求。其他内容，例如 Reasoning Control、缓存行为、安全设置、媒体细节、Built-in Tool、有状态 Conversation 与 Provider 特定采样，都应作为类型化扩展表示；若不支持则明确拒绝。
The layer may be a hosted aggregator, managed gateway, self-hosted proxy, or SDK. Product shape determines where keys, data, telemetry, billing, configuration, and operations live. A hosted service reduces infrastructure work but becomes part of the data and availability path. A self-hosted proxy increases control but adds databases, upgrades, scaling, incident response, and reconciliation. An SDK avoids a network hop but distributes policy across every application release.
统一层可以是托管聚合器、托管 Gateway、自托管 Proxy 或 SDK。产品形态决定密钥、数据、遥测、计费、配置与运维位于何处。托管服务减少基础设施工作，却会进入数据与可用性路径；自托管 Proxy 增加控制力，也带来数据库、升级、扩缩容、事故响应与对账责任；SDK 避免额外网络跳转，但会把策略分散到每个应用版本中。
Unified API building blocks统一 API 构建块
| Building block构建块 | Best fit最适合 | Verify before choosing选择前验证 |
|---|---|---|
| One base URL and auth单一基础地址（Base URL）与认证 | Applications use one service identity and model access boundary.应用使用一个服务身份与模型访问边界。 | Define tenant isolation, provider keys, BYOK, rotation, least privilege and direct rollback.定义租户隔离、供应商密钥、BYOK、轮换、最小权限与直连回滚。 |
| Model catalog and flags模型目录与 Flag | Stable aliases map to versioned capabilities and eligible endpoints.稳定别名映射到版本化能力与合格端点。 | Source, verify and refresh context, modalities, parameters, region, policy, price and lifecycle.为上下文、模态、参数、区域、策略、价格与生命周期标注来源、验证并刷新。 |
| Provider adapters供应商适配器 | Translate canonical requests, responses, streams, errors and usage.转换规范请求、响应、流、错误与用量。 | Contract-test every required feature and preserve provider-specific escape hatches.对每个所需功能做契约测试，并保留供应商特定出口。 |
| Routing and fallback路由与回退 | Select among eligible models or providers and recover from bounded failures.在合格模型或供应商间选择，并从有边界故障恢复。 | Record candidate set, decision, policy, attempts, substitutions, charges and final error.记录候选集合、决策、策略、尝试、替换、费用与最终错误。 |
| Usage and evidence用量与证据 | Normalize operational fields for observability, billing and governance.为可观测、计费与治理标准化运营字段。 | Store native units and IDs beside normalized data to reconcile providers and incidents.把原生单位与 ID 同标准化数据并存，以核对供应商与事故。 |
Normalize transport, preserve semantics统一传输，保留语义
Messages, common tools, model alias, timeout, trace, tenant, budget and required capability flags.
Provider or modality-specific fields isolated behind documented namespaces and feature detection.
Canonical request ID plus native provider IDs, transformations, route, attempts, stream, errors, usage and cost.
Reject missing capabilities before execution; never silently ignore required fields or downgrade semantics.
Message、常用工具、模型别名、超时、调用链、租户、预算与所需能力标记。
供应商或模态特定字段隔离在有文档的命名空间与功能检测后。
规范请求 ID 加原生供应商 ID、转换、路由、尝试、流、错误、用量与成本。
执行前拒绝缺失能力；不得静默忽略必需字段或降级语义。
Migrate to a unified API safely安全迁移到统一 API
Start with one bounded workload instead of switching the entire application. Choose a flow whose output schema and quality can be measured, then preserve its native implementation as the baseline. The migration is successful only if the unified route preserves required behavior, operational evidence, privacy terms, and rollback—not merely if the first response returns text.
不要一次切换整个应用，应先选择一个边界清晰的工作负载。优先选择输出 Schema 与质量可测量的流程，并保留原生实现作为基线。只有统一路由同时保留必需行为、运营证据、隐私条款与回滚能力，迁移才算成功，而不是“第一次请求返回了文本”就算完成。
- Inventory models, endpoint families, SDK assumptions, features, provider extensions, keys, data terms, errors, quotas, and billing dimensions.
- Define required capability flags and fail closed before routing when no candidate satisfies them.
- Build golden contract tests for ordinary, streaming, structured-output, tool, media, timeout, rate-limit, and provider-failure paths.
- Dual-run unified and native paths; diff outputs, events, errors, usage, latency, retries, selected route, and cost.
- Verify tenant isolation, secret rotation, log redaction, retention, regional routing, native request IDs, and invoice reconciliation.
- Canary by workload and tenant, preserve direct rollback, and block deployment on required-invariant regressions.
- 盘点模型、端点家族、SDK 假设、功能、Provider 扩展、密钥、数据条款、错误、配额与计费维度。
- 定义必需能力 Flag；没有候选满足时，应在路由前关闭失败，而不是静默降级。
- 为普通、流式、结构化输出、工具、媒体、超时、限流与 Provider 故障路径建立 Golden 契约测试。
- 双轨运行统一与原生路径，对比输出、事件、错误、用量、延迟、重试、所选路由与成本。
- 验证租户隔离、Secret 轮换、日志脱敏、保留策略、区域路由、原生 Request ID 与账单对账。
- 按工作负载和租户灰度，保留直连回滚，并在必需不变量回归时阻止部署。
Use a canonical envelope with typed adapters使用规范封装与类型化适配器
The canonical envelope contains identity, trace, workload alias, messages, required capabilities, timeout, budget and policy. A catalog resolves eligible endpoints before translation. Typed adapters map only supported semantics. The response includes normalized output plus route and native evidence. Escape hatches are explicit, scoped and covered by tests.
规范封装包含身份、调用链、工作负载别名、Message、所需能力、超时、预算与策略。目录在转换前解析合格端点；类型化适配器只映射受支持语义；响应包含标准化输出以及路由和原生证据；出口应显式、受限并被测试覆盖。
Production rule: a unified API should make common operations portable without erasing the differences that determine correctness.
生产规则：统一 API 应让通用运营可移植，但不能抹去决定正确性的差异。
A unified LLM API is still an inference interface统一 LLM API 仍是推理接口
The unified API gives an agent consistent access to models. QVeris gives that agent consistent discovery and execution of external APIs, tools, services and live data. Stack them as inference interface plus capability interface, with shared workflow trace context.
统一 API 让智能体一致访问模型；QVeris 让智能体一致发现和执行外部 API、工具、服务与实时数据。把两者作为推理接口与能力接口叠加，并共享工作流调用链上下文。
FAQ
One client contract that accesses multiple models or providers through shared authentication, requests, routing and operational evidence.
No. It simplifies transport and operations; capabilities, behavior, quality, policy and price remain model- and endpoint-specific.
For simple compatible calls, sometimes. Production workloads still require conformance, dual-run, canary and rollback.
通过共享认证、请求、路由与运营证据访问多个模型或供应商的单一客户端契约。
不会。它简化传输与运营；能力、行为、质量、策略与价格仍因模型和端点而异。
简单兼容调用有时可以；生产负载仍需一致性测试、双轨、灰度与回滚。