Model Router vs Gateway
The Router Chooses; the Gateway Executes模型路由器与网关：前者做选择，后者负责治理与执行
A model router is a decision component. A model gateway is the operational control plane around the request. They overlap in routing language, but solve different scopes and often belong in one architecture.
模型路由器是决策组件；模型网关是请求周围的运营控制平面。两者在路由术语上有交集，但解决范围不同，且经常同属一个架构。
TL;DR
It filters, scores and selects an eligible model or endpoint for the workload.
It authenticates, adapts, limits, executes, observes and governs the request lifecycle.
Routing can be a policy module inside the broader gateway control plane.
The selected candidate, reasons, attempts, transformations and result belong in one trace.
它为工作负载过滤、评分并选择合格模型或端点。
它对请求生命周期执行认证、适配、限流、执行、观测与治理。
路由可以作为更广泛网关控制平面中的策略模块。
所选候选、原因、尝试、转换与结果应属于同一调用链。
Decision plane versus execution plane决策平面与执行平面
The model router receives a versioned workload contract, candidate catalog, tenant policy and current operational signals. It removes ineligible endpoints, scores the remainder and emits a route plan with the selected endpoint, adapter, fallback limit and rationale. Its core output is a reproducible decision—not a network response from a model.
Model Router 接收版本化工作负载契约、候选目录、租户策略与当前运营信号，移除不合格端点、为剩余项评分，并输出包含所选端点、Adapter、Fallback 上限与理由的 Route Plan。它的核心输出是可复现决策，而不是来自模型的网络响应。
The gateway owns the execution boundary: caller identity, credentials, provider adapters, deadlines, quotas, retries, cache, policy enforcement, telemetry and final response. It validates the route plan before use because the execution layer must not trust a stale or compromised decision to bypass hard constraints. A thin gateway may use a static router; a richer gateway may host workload-specific policies.
Gateway 拥有执行边界：调用方身份、凭证、Provider Adapter、截止时间、配额、重试、缓存、策略执行、Telemetry 与最终响应。它在使用前必须验证 Route Plan，不能让过期或被破坏的决策绕过硬约束。轻量 Gateway 可使用静态 Router，更完整的 Gateway 则可承载面向不同工作负载的策略。
Use a standalone router when several execution systems need the same decision service or when routing evolves independently. Embed it when one gateway owns the entire lifecycle and low latency matters. In both cases, keep the plan contract explicit so teams can test selection separately from provider execution.
当多个执行系统需要同一决策服务，或路由策略独立演进时，可使用独立 Router；当一个 Gateway 拥有完整生命周期且低延迟重要时，可将其嵌入。无论哪种方式，都要保持 Plan Contract 明确，让团队能够分别测试选择与 Provider Execution。
Router and gateway responsibilities路由器与网关职责
| Responsibility职责 | Best fit最适合 | Verify before choosing选择前验证 |
|---|---|---|
| Candidate eligibility候选合格性 | Router filters by capability, policy, region and budget.路由器按能力、策略、区域与预算过滤。 | Gateway must supply a versioned catalog and enforce the resulting constraints.网关必须提供版本化目录并执行所得约束。 |
| Scoring and selection评分与选择 | Router compares quality, cost, latency, health or preference signals.路由器比较质量、成本、延迟、健康或偏好信号。 | Gateway records the policy version and selected endpoint before execution.网关在执行前记录策略版本与所选端点。 |
| Provider translation供应商转换 | Router may declare an adapter requirement.路由器可以声明适配器要求。 | Gateway maps request, stream, response, error and usage semantics.网关映射请求、流、响应、错误与用量语义。 |
| Operational controls运营控制 | Router can consider quota and health as inputs.路由器可把配额与健康作为输入。 | Gateway owns auth, keys, rate limits, cache, retries, circuit breakers and cancellation.网关拥有认证、密钥、限流、缓存、重试、熔断与取消。 |
| Evidence and governance证据与治理 | Router explains candidate exclusions and score.路由器解释候选排除项与得分。 | Gateway joins route evidence with attempts, native IDs, usage, policy and final disposition.网关把路由证据与尝试、原生 ID、用量、策略及最终处置连接。 |
Define a clean component contract定义清晰组件契约
Workload contract, eligible catalog snapshot, policy version and fresh operational signals.
Selected endpoint, ranked alternates, exclusions, reason codes and adapter requirements.
Authorize identity, obtain credentials, translate, enforce budgets and execute bounded attempts.
One parent trace connects decision, execution, provider response and downstream tool work.
工作负载契约、合格目录快照、策略版本与新鲜运营信号。
所选端点、排序备用项、排除项、原因码与适配器要求。
授权身份、获取凭证、转换、执行预算并运行有界尝试。
一个父调用链连接决策、执行、供应商响应与下游工具工作。
Test the router and gateway together联合测试路由器与网关
Router unit tests prove deterministic selection from fixed inputs. End-to-end tests prove that the selected route can actually authenticate, preserve the workload contract and return within budget. You need both: a perfect decision is useless if the gateway cannot execute it, while successful execution should not excuse an unexplainable choice.
Router 单元测试证明固定输入能够产生确定性选择；端到端测试证明所选路由能够真实认证、保留工作负载契约并在预算内返回。两者缺一不可：Gateway 无法执行时，再完美的决策也无用；执行成功也不能掩盖无法解释的选择。
- Freeze workload, catalog, policy and health fixtures so route decisions are reproducible.
- Test hard exclusions before score weights, preferences and tie-breaking.
- Verify every selected endpoint has a valid adapter, scoped credential and policy-approved network path.
- Inject stale health, quota exhaustion, malformed plans, adapter errors and execution failures.
- Confirm fallback re-enters selection with the failed route excluded and the same parent budget.
- Verify the trace explains candidates, exclusions, choice, attempts, validation and final outcome.
- 冻结工作负载、目录、策略与健康 Fixture，使路由决策可复现。
- 先测试硬排除，再测试评分权重、偏好和同分规则。
- 验证每个所选端点都有有效 Adapter、受限凭证与策略批准的网络路径。
- 注入过期健康、配额耗尽、异常 Plan、Adapter 错误和执行故障。
- 确认 Fallback 排除失败路由后重新进入选择，并继续使用同一父预算。
- 确认 Trace 能解释候选、排除、选择、尝试、验证与最终结果。
Embed the router, separate the contract嵌入路由器，同时分离契约
The gateway constructs a routing context from caller identity, workload requirements, catalog and policy versions, remaining budget and fresh health state. The router returns a versioned route plan plus the candidate set, exclusions and score inputs. Even when implemented in the same process, pass this through a typed internal contract rather than sharing mutable global state.
Gateway 根据调用方身份、工作负载要求、目录与策略版本、剩余预算和新鲜健康状态构建 Routing Context。Router 返回版本化 Route Plan，以及候选集合、排除项与评分输入。即使二者在同一进程中实现，也应通过 Typed Internal Contract 传递，而不是共享可变全局状态。
The gateway validates plan age and hard constraints, resolves credentials and adapters, executes one bounded attempt and validates the output. It records both decision evidence and provider-native execution evidence under one trace. This separation lets routing policy evolve without fragmenting authentication, safety, accounting or telemetry.
Gateway 验证 Plan 新鲜度与硬约束，解析凭证和 Adapter，执行一次有界尝试并验证输出。决策证据与供应商原生执行证据都记录在同一 Trace 下。这样路由策略可以演进，而不会拆散认证、安全、核算或 Telemetry。
Production rule: the gateway must independently enforce every hard constraint the router used to choose a candidate.
生产规则：Gateway 必须独立执行 Router 选择候选时使用的每个硬约束。
Model routing does not route external capabilities模型路由不等于外部能力路由
A router chooses inference; a gateway governs inference execution. QVeris complements both by discovering and calling external APIs, tools, services and live data. Keep capability requirements in the workload contract so the model route supports the required tool shape, but inspect capability schema, permissions, health and freshness independently.
Router 选择推理，Gateway 治理推理执行；QVeris 通过发现并调用外部 API、工具、服务与实时数据补充两者。能力需求应写入工作负载契约，确保模型路由支持所需工具结构；但能力 Schema、权限、健康状态与新鲜度仍要独立检查。
Connect model and tool calls to the same parent workflow trace while keeping separate credentials, action IDs, retries and fallback pools. A different model may reuse a verified tool result, but it must not automatically gain permission to repeat a consequential action or switch to a semantically different data source.
模型与工具调用可连接到同一父 Workflow Trace，但凭证、Action ID、重试与 Fallback Pool 应分别维护。不同模型可以复用已验证工具结果，却不能自动获得重复重要动作的权限，也不能自动切换到语义不同的数据源。
FAQ
No. It can be a library, plugin, policy module or independent service.
Yes. It can govern one fixed upstream while still providing auth, limits, adapters and evidence.
The execution layer, usually the gateway, because it knows attempts, deadlines and side effects.
不是。它可以是库、插件、策略模块或独立服务。
能。即使只连接固定上游，它仍可提供认证、限制、适配与证据。
执行层，通常是网关，因为它知道尝试、截止时间与副作用。