LiteLLM vs Kong AI Gateway: Focused Model Proxy or Enterprise API Platform?LiteLLM 与 Kong AI 网关：模型代理，还是企业 API 平台？
This is less a feature race than a scope decision. LiteLLM focuses on model-provider normalization and gateway control. Kong extends an established API gateway with AI-aware plugins across identity, security, observability, hybrid deployment, Kubernetes, MCP, and legacy APIs.
这与其说是功能竞赛，不如说是平台范围决策。LiteLLM 聚焦模型供应商标准化与网关控制；Kong 则在成熟 API 网关上扩展 AI 感知插件，并覆盖身份、安全、可观测性、混合部署、Kubernetes、MCP 和传统 API。
TL;DR
Choose it when the main job is multi-provider model access, virtual keys, budgets, fallbacks, and a self-operated AI proxy.
Choose it when AI traffic must join an existing estate of services, identities, plugins, policies, deployment modes, and API operations.
A larger platform can reduce tool sprawl but increases control-plane, plugin, licensing, configuration, and operator decisions.
If Kong already governs critical APIs, extending the same policy plane may be simpler than introducing a separate AI gateway.
当核心任务是多供应商模型访问、虚拟密钥、预算、回退和自主运营 AI 代理时更合适。
当 AI 流量必须接入现有服务、身份、插件、策略、部署模式和 API 运营体系时更合适。
更大平台可以减少工具碎片，但会增加控制面、插件、许可、配置和运营决策。
如果 Kong 已治理关键 API，扩展同一策略平面可能比引入独立 AI 网关更简单。
Decide whether AI is a separate plane先决定 AI 是否需要独立平面
LiteLLM treats model access as a specialized gateway problem. Its proxy normalizes providers behind an OpenAI-style interface and adds routing, virtual keys, budgets, spend tracking, caching, guardrails, and integrations.
LiteLLM 把模型访问视为专门的网关问题。其代理通过 OpenAI 风格接口标准化供应商，并增加路由、虚拟密钥、预算、支出追踪、缓存、护栏和集成。
Kong treats AI as another governed traffic class inside an API platform. Official documentation describes AI Proxy and AI Proxy Advanced plus plugins for semantic routing, caching, prompt controls, token rate limiting, observability, guardrails, MCP and A2A traffic, with Konnect, self-hosted, hybrid, DB-less, and Kubernetes modes.
Kong 把 AI 视为 API 平台中的另一类受治理流量。官方文档介绍 AI 代理/Advanced，以及语义路由、缓存、提示词控制、Token 限流、可观测性、护栏、MCP 与 A2A 插件，并支持 Konnect、自托管、混合、DB-less 和 Kubernetes 模式。
LiteLLM vs Kong AI Gateway side by sideLiteLLM 与 Kong AI 网关并排比较
| Decision surface决策面 | LiteLLM | Kong AI Gateway |
|---|---|---|
| Platform scope平台范围 | AI model proxy, SDK and gateway governanceAI 模型代理、SDK 与网关治理 | Enterprise API gateway with AI-aware plugins带 AI 感知插件的企业 API 网关 |
| Traffic estate流量体系 | Primarily model-provider traffic主要是模型供应商流量 | AI, REST, gRPC, services, MCP, A2A and legacy APIsAI、REST、gRPC、服务、MCP、A2A 与传统 API |
| Policy model策略模型 | Virtual keys, teams, budgets and model/provider rules虚拟密钥、团队、预算与模型/供应商规则 | Services, routes, consumers, groups, plugins and control planesService、路由、Consumer、Group、Plugin 与控制面 |
| Deployment部署 | Self-operated proxy and surrounding stores自主运营代理与周边存储 | Managed, traditional, hybrid, DB-less and Kubernetes options托管、传统、混合、DB-less 与 Kubernetes |
| Extensibility扩展性 | Callbacks and AI integrations回调与 AI 集成 | Large plugin system and custom plugin development大型插件体系与自定义插件开发 |
| Best fit最佳适配 | Teams needing a dedicated model access plane需要独立模型访问平面的团队 | Organizations standardizing all API and AI traffic统一标准化全部 API 与 AI 流量的组织 |
Let platform gravity decide让平台重力决定
The team needs an AI-specific proxy quickly, wants direct control of provider routing and budgets, and does not need a general API platform.
Existing Kong operations, identity, plugins, hybrid networking, Kubernetes, compliance, and non-AI APIs should remain on one control plane.
The architecture adds a second gateway without a clear policy boundary, or required AI plugins, editions, and deployment modes are not validated.
团队需要快速建立 AI 专用代理、直接控制供应商路由与预算，且不需要通用 API 平台。
现有 Kong 运营、身份、插件、混合网络、Kubernetes、合规与非 AI API 应保留在一个控制面。
架构增加第二个网关却没有明确策略边界，或所需 AI 插件、版本与部署模式尚未验证。
Avoid an accidental gateway chain避免无意形成网关链
Map client, edge, API gateway, AI gateway, model provider, telemetry and tool calls as one request path. If both LiteLLM and Kong remain, decide whether Kong owns enterprise ingress while LiteLLM owns model normalization, then prevent duplicate authentication, rate limits, retries, caching and content policy. Preserve one end-to-end trace ID across both hops.
把客户端、边缘、API 网关、AI 网关、模型供应商、遥测和工具调用画成一条请求路径。如果两者都保留，应明确 Kong 负责企业入口、LiteLLM 负责模型标准化，并防止认证、限流、重试、缓存和内容策略重复；全链路保留一个调用链 ID。
Architecture rule: each synchronous hop must have one reason to exist and one failure owner.
架构规则：每个同步跳点都必须有唯一存在理由和唯一故障负责人。
A platform-scope proof平台范围验证
- Inventory existing gateways, identities, services, plugins, certificates, policies, regions, and on-call ownership.
- Run identical model traffic through both with tools, streaming, structured output, long context, and provider failures.
- Test one enterprise identity flow, token budget, prompt guard, audit export, Kubernetes rollout, and rollback.
- Measure p99 latency, policy correctness, duplicate calls, configuration lead time, and incident diagnosis time.
- Price licenses, control planes, data planes, stores, telemetry, platform labor, and upgrade windows.
- 盘点现有网关、身份、服务、插件、证书、策略、区域和值班责任。
- 用工具调用、流式、结构化输出、长上下文与供应商故障在两者上运行相同模型流量。
- 测试一次企业身份流、Token 预算、提示词护栏、审计导出、Kubernetes 发布和回滚。
- 测量 p99 延迟、策略正确率、重复调用、配置交付时间和事故诊断时间。
- 计算许可、控制面、数据面、存储、遥测、平台人力和升级窗口成本。
Migrate by policy domain按策略域迁移
Create a matrix for identity, consumer, key, route, model, provider, budget, rate limit, retry, fallback, cache, guardrail, logging, trace and audit requirements. Map each requirement to one target system and edition. Mirror traffic, compare semantics and policy outcomes, then move one consumer group at a time with a tested rollback.
为身份、Consumer、密钥、路由、模型、供应商、预算、限流、重试、回退、缓存、护栏、日志、追踪和审计建立矩阵；把每项要求映射到唯一目标系统与版本。镜像流量并比较语义和策略结果，再按 Consumer Group 逐步迁移并保留已验证回滚。
The API platform still needs capability discoveryAPI 平台仍需要能力发现
LiteLLM or Kong controls model and API traffic. QVeris complements them by helping an agent discover which external API, dataset, or tool can satisfy an intent, then invoke it under governed credentials and contracts. Link model, gateway and capability traces into one action record.
LiteLLM 或 Kong 控制模型与 API 流量；QVeris 作为互补层，帮助智能体发现哪个外部 API、数据集或工具能满足意图，并在受治理凭证与契约下调用。应把模型、网关和能力追踪连接为一条动作记录。
FAQ
Kong documents AI Gateway as AI plugins and capabilities used with Kong Gateway across supported deployment modes.
It provides virtual keys, budgets, spend tracking, routing, guardrails and related controls; validate the required depth and support model.
Yes, but assign enterprise ingress and model normalization deliberately, and eliminate duplicated synchronous behavior.
It depends on existing skills and estate. An already-operated Kong platform may be simpler; a new Kong program may be heavier than a focused proxy.
Kong 文档把 AI 网关描述为在 Kong 网关支持的部署模式中使用的一组 AI 插件与能力。
它提供虚拟密钥、预算、支出追踪、路由、护栏等控制；仍需验证所需深度与支持模式。
可以，但要有意划分企业入口与模型标准化，并消除重复同步行为。
取决于现有能力与体系。已运营 Kong 的组织可能更简单；新建 Kong 体系可能比聚焦代理更重。