AI Gateway vs API Gateway
Shared Controls, Different Semantics
AI 网关与 API 网关：共享控制，不同语义
Both gateways sit on request paths, but they optimize different contracts. An API gateway governs general services; an AI gateway understands model-specific requests, streams, tokens, tools and routing.
两类网关都位于请求路径上，但优化不同契约。API 网关治理通用服务；AI 网关理解模型特定请求、流、Token、工具与路由。
TL;DR
They expose, authenticate, rate-limit, route and observe REST, RPC and other service traffic.
They add model catalogs, provider adapters, token-aware limits, streams, tools and model routing.
Identity, policy, traffic control, transforms and observability matter to both.
Keep the enterprise service edge and add an AI-aware control plane where model semantics begin.
它暴露、认证、限流、路由并观测 REST、RPC 与其他服务流量。
它增加模型目录、供应商适配器、Token 感知限制、流、工具与模型路由。
身份、策略、流量控制、转换与可观测对两者都重要。
保留企业服务边界，并在模型语义开始处增加 AI 感知控制平面。
The difference is the upstream contract 区别在于上游契约
A general API gateway treats the upstream as a service contract: method, path, schema, identity, policy, traffic and response. An AI gateway must also understand model families, provider-specific adapters, prompts, context, tokens, streams, tool calls, structured output, safety controls, semantic cache and routing evidence.
通用 API 网关把上游视为服务契约：Method、Path、结构定义、身份、策略、流量与响应。AI 网关还必须理解模型家族、供应商适配器、提示词、上下文、Token、流、工具调用、结构化输出、安全控制、语义缓存与路由证据。
Product boundaries vary. Some API gateways add AI plugins; some AI gateways inherit general gateway controls. Evaluate the actual request path and feature ownership instead of relying on the product label.
产品边界并不统一。有些 API 网关添加 AI 插件；有些 AI 网关继承通用网关控制。应评估实际请求路径与功能归属，而不是依赖产品标签。
Gateway responsibilities side by side 网关职责并排对比
| Dimension 维度 | Best fit 最适合 | Verify before choosing 选择前验证 |
|---|---|---|
| Primary upstream 主要上游 | API gateway: services, microservices and third-party APIs. API 网关：服务、微服务与第三方 API。 | AI gateway: model providers, deployments and inference endpoints. AI 网关：模型供应商、Deployment 与推理端点。 |
| Traffic units 流量单位 | Requests, bytes, connections and service quotas. 请求、字节、连接与服务配额。 | Requests plus tokens, context, streams, model quotas and workload budgets. 请求加 Token、上下文、流、模型配额与工作负载预算。 |
| Routing semantics 路由语义 | Path, host, version, region, health and service policy. Path、主机地址、版本、区域、健康与服务策略。 | Capability, model, quality, cost, latency, safety, fallback and provider policy. 能力、模型、质量、成本、延迟、安全、故障切换与供应商策略。 |
| Transforms and cache 转换与缓存 | Schema and protocol transforms; key-based response cache. 结构定义与协议转换；基于密钥的响应缓存。 | Provider adapters, prompt or tool transforms and optional semantic cache. 供应商适配器、提示词或工具转换及可选语义缓存。 |
| Evidence 证据 | Service request IDs, traces, status, latency and policy decisions. 服务请求 ID、调用链、状态、延迟与策略决策。 | Adds model route, attempts, tokens, tool events, native IDs, usage and cost evidence. 增加模型路由、尝试、Token、工具事件、原生 ID、用量与成本证据。 |
Choose the boundary by responsibility 按职责选择边界
Use the API gateway for organization-wide ingress, identity and generic service governance.
Use the AI gateway where model-aware translation, routing and evidence are required.
Assign one owner for authentication, retry, transforms, limits and response changes at each layer.
Propagate identity, tenant, request ID and trace context across both layers.
用 API 网关承担组织级入口、身份与通用服务治理。
在需要模型感知转换、路由与证据处使用 AI 网关。
为每层认证、重试、转换、限制与响应变化指定唯一负责人。
在两层间传播身份、租户、请求 ID 与调用链上下文。
Validate a two-gateway request path 验证双网关请求路径
- Draw the exact hop order, trust boundaries, key ownership and data exposure.
- Assign one owner for retries, rate limits, cache, transforms and final errors.
- Test streaming, cancellation, tool calls, large payloads and partial failures end to end.
- Correlate service traces with model route, attempt, token and native provider evidence.
- Load-test combined latency, backpressure and failure amplification.
- 绘制准确 Hop 顺序、信任边界、密钥所有权与数据暴露。
- 为重试、限流、缓存、转换与最终错误指定一个负责人。
- 端到端测试流、取消、工具调用、大请求数据与局部故障。
- 把服务调用链与模型路由、尝试、Token 及原生供应商证据关联。
- 压测组合延迟、背压与故障放大。
Layer generic ingress and model-aware control 分层通用入口与模型感知控制
Clients enter through the API gateway for enterprise identity, exposure policy and service routing. AI traffic then reaches an AI gateway that resolves capabilities, adapts provider contracts, selects endpoints and records inference evidence. Shared trace context crosses both; keys, retries, cache and mutations have explicit owners.
客户端先通过 API 网关处理企业身份、暴露策略与服务路由；AI 流量随后进入 AI 网关，解析能力、适配供应商契约、选择端点并记录推理证据。两层共享调用链上下文；密钥、重试、缓存与修改都有明确负责人。
Production rule: adding a second gateway is safe only when ownership and failure behavior are explicit.
生产规则：只有明确职责归属与故障行为时，增加第二层网关才安全。
QVeris is the capability layer above the model path QVeris 是模型路径之上的能力层
Neither gateway category automatically discovers and executes the external APIs, tools, services and live data an agent needs. QVeris complements the model path with Discover → Inspect → Call. A practical stack can be API gateway for service ingress, AI gateway for inference and QVeris for external capabilities.
两类网关都不会自动发现并执行智能体所需的外部 API、工具、服务与实时数据。QVeris 以 Discover → Inspect → 调用补充模型路径。实际技术栈可以是：API 网关管服务入口，AI 网关管推理，QVeris 管外部能力。
A Production Decision Scorecard for AI Gateway vs API GatewayAI Gateway 与 API Gateway的生产决策评分卡
For AI Gateway vs API Gateway, the useful question is not which product has more checkmarks. It is which design gives the team the right boundary, evidence, operating model, and exit path for a defined workload.
针对“AI 网关 vs API 网关”，真正有价值的问题不是哪款产品拥有更多勾选项，而是哪种设计能为明确工作负载提供正确边界、证据、运营模式和退出路径。
Map model semantics, token and context controls, provider routing, prompt-aware policy, and generic API ingress responsibilities. Decide which component owns each decision, where policy is enforced, and whether the products are substitutes, complements, or overlapping layers.
梳理模型语义、Token 与上下文控制、供应商路由、提示词感知策略，以及通用 API 入口责任。明确每项决策由哪个组件负责、策略在哪里执行，以及两者究竟是替代、互补还是部分重叠。
To validate AI Gateway vs API Gateway, replay simple, long-context, streaming, structured-output, tool-calling, high-concurrency, and failure cases. Measure accepted-result quality, completion, p50 and tail latency, retries, trace completeness, and effective cost.
验证“AI 网关 vs API 网关”时，重放简单、长上下文、流式、结构化输出、工具调用、高并发和失败案例，衡量合格结果质量、完成率、常规与长尾延迟、重试、追踪完整性和实际成本。
When evaluating AI Gateway vs API Gateway, include hosting, regional capacity, data retention, identity integration, policy maintenance, upgrades, incident response, support, compliance evidence, and the custom adapters the team must keep current.
评估“AI 网关 vs API 网关”时，纳入托管、区域容量、数据留存、身份集成、策略维护、升级、事故响应、支持、合规证据，以及团队必须持续维护的自定义适配器。
Before rolling out AI Gateway vs API Gateway, version a neutral request and evidence envelope, shadow traffic, classify semantic differences, preserve trace identity, and prove a staged rollback. Prefer the option that keeps policy and workload contracts portable.
上线“AI 网关 vs API 网关”前，版本化中立请求与证据封装，运行影子流量，分类语义差异，保留追踪身份，并证明可分阶段回滚。优先选择能让策略和工作负载契约保持可迁移的方案。
FAQ
Yes, but generic proxying does not automatically provide model-aware semantics or evidence.
Yes. Verify the actual plugins, deployment, control ownership and supported model features.
Usually no. Put bounded retry ownership in one layer to avoid amplification and duplicate work.
能，但通用代理不会自动提供模型感知语义或证据。
能。仍需验证实际插件、部署、控制归属与支持的模型功能。
通常不应。把有界重试归属放在一层，避免放大与重复工作。