OpenAI API Proxy
Architecture, Security, and SetupOpenAI API 代理
架构、安全与生产部署
An OpenAI API proxy protects upstream credentials and gives applications one controlled server-side endpoint. This guide explains the request path, trust boundary, streaming behavior, security controls, implementation pattern, and production tests.
OpenAI API 代理用于保护上游凭证，并为应用提供统一、受控的服务端入口。本指南说明完整请求路径、信任边界、流式行为、安全控制、实施方式与生产验证方法。
TL;DR
Direct answer: an OpenAI API proxy is a server-side intermediary between applications and the OpenAI API. It authenticates the caller, validates and limits the request, attaches a protected upstream credential, forwards approved traffic, preserves streaming, and returns the response while recording only the evidence the operator intentionally retains.
直接答案：OpenAI API 代理是位于应用与 OpenAI API 之间的服务端中间层。它认证调用方、验证并限制请求、附加受保护的上游凭证、转发获准流量、保持流式传输，并在返回响应的同时，只记录运营方明确决定保留的证据。
A proxy removes the provider key from clients, but it still needs its own user or workload authentication, authorization, abuse controls, and quotas.
The proxy can see prompts, files, tool arguments, outputs, usage, and upstream credentials. Its operators, logs, plugins, and dependencies enter the security model.
Streaming events, cancellation, native request IDs, errors, usage, tool calls, and structured output should not be silently changed.
Version every mutation, name one retry owner, define retention, test failure paths, and maintain an emergency rollback or alternate route.
代理把供应商密钥移出客户端，但仍需实现自己的用户或工作负载认证、授权、滥用防护与配额。
代理能够看到提示词、文件、工具参数、输出、用量和上游凭证，因此运营人员、日志、插件与依赖都属于安全模型的一部分。
流式事件、取消、原生请求 ID、错误、用量、工具调用与结构化输出不应被静默修改。
所有修改都要有版本，重试只能有一个责任方，数据保留必须明确，故障路径要经过测试，并保留紧急回滚或备用路径。
What an OpenAI API proxy is—and is notOpenAI API 代理是什么，又不是什么
At its simplest, the proxy exposes a server-side endpoint shaped like the OpenAI API, receives a request from an approved application, and forwards it to the configured OpenAI endpoint with a credential stored on the server. This solves a narrow but important problem: untrusted clients no longer receive or transmit the upstream secret.
最简单的 OpenAI API 代理会在服务端暴露一个与 OpenAI API 形状相近的端点，接收获准应用的请求，再使用保存在服务端的凭证转发到预先配置的 OpenAI 端点。它首先解决一个狭窄但重要的问题：不可信客户端不再持有或传输上游密钥。
A proxy does not automatically make an integration secure. If it accepts requests without authenticating users, allows arbitrary models or upstream URLs, permits unbounded bodies, logs sensitive content, or exposes detailed upstream errors, it can move the key while creating a larger abuse surface. It also does not automatically become an AI gateway: routing, budgets, guardrails, caching, provider translation, and multi-tenant governance are optional capabilities.
代理并不会自动让集成变安全。如果它不认证用户、允许任意模型或上游地址、接受无限大小的请求正文、记录敏感内容，或者原样暴露上游详细错误，那么它只是移动了密钥，却扩大了滥用面。代理也不天然等同于 AI 网关；路由、预算、护栏、缓存、供应商协议转换与多租户治理都属于额外能力。
The production request flow, step by step生产请求如何一步步通过代理
| Step步骤 | Proxy responsibility代理职责 | Evidence证据 |
|---|---|---|
| 1 | Terminate TLS; reject invalid method, path, content type, and oversized body终止 TLS，并拒绝错误方法、路径、Content-Type 与超大正文 | Request ID, route, rejection category请求 ID、路由与拒绝类别 |
| 2 | Authenticate the user or workload and authorize the requested operation认证用户或工作负载，并授权本次操作 | Actor, tenant, policy version, decision主体、租户、策略版本与决策 |
| 3 | Validate the payload, model alias, tools, limits, and metadata验证请求正文、模型别名、工具、限制与元数据 | Schema version, alias, matched limitSchema 版本、别名与命中限制 |
| 4 | Resolve a server-side endpoint and credential; never trust an upstream URL or key from the client在服务端解析上游端点与凭证，绝不信任客户端提供的 URL 或密钥 | Route ID and secret reference, not secret value路由 ID 与密钥引用，不记录密钥值 |
| 5 | Forward with a total deadline, preserve streaming and cancellation, and apply one bounded retry policy设置总截止时间，保持流式与取消语义，并执行单一、有限的重试策略 | Attempts, timing stages, bytes or events delivered尝试次数、各阶段耗时与已传输字节/事件 |
| 6 | Return stable client errors while retaining native IDs, model, usage, and finish state for diagnosis返回稳定客户端错误，同时保留原生 ID、模型、用量与结束状态以便诊断 | Upstream request ID, status, usage, final outcome上游请求 ID、状态、用量与最终结果 |
Proxy, gateway and compatibility endpoint代理、网关与兼容端点
A transparent proxy may forward the OpenAI contract with minimal change. A compatibility proxy may translate requests to other upstreams. A gateway adds broader identity, policy, quotas, routing, cache and observability. Products can combine these roles, so the important question is what code touches the request and why.
透明代理可能几乎不修改地转发 OpenAI 契约；兼容代理可能把请求转换给其他上游；网关则增加更广泛的身份、策略、配额、路由、缓存与可观测。产品可组合这些角色，因此关键问题是哪些代码为何触碰请求。
The proxy becomes a high-value secret and data boundary. Evaluate who operates it, how credentials are stored, where payloads flow, what is logged, which fields are modified, and what happens when the proxy or upstream fails.
代理成为高价值密钥与数据边界。需评估谁运营、凭证如何存储、请求数据流向哪里、记录什么日志、修改哪些字段，以及代理或上游失败时会发生什么。
Proxy responsibilities and risks代理职责与风险
| Control控制 | Best fit最适合 | Verify before choosing选择前验证 |
|---|---|---|
| Key isolation密钥隔离 | Keep provider keys out of clients and scope proxy identities.让供应商密钥离开客户端，并约束代理身份。 | Secret concentration increases blast radius; use least privilege, rotation and audit.密钥集中扩大爆炸半径；采用最小权限、轮换与审计。 |
| Validation and policy验证与策略 | Reject malformed, unauthorized or disallowed requests before upstream execution.在上游执行前拒绝异常、未授权或禁止请求。 | Define fail-open versus fail-closed behavior and preserve error evidence.定义 Fail-open 与 Fail-closed 行为，并保留错误证据。 |
| Logging and observability日志与可观测 | Correlate clients, requests, upstream IDs, latency, usage and failures.关联客户端、请求、上游 ID、延迟、用量与故障。 | Redact sensitive content, control retention and limit operator access.脱敏敏感内容、控制保留并限制运维访问。 |
| Retries and limits重试与限制 | Protect upstream capacity and recover from approved transient failures.保护上游容量并从批准的瞬态故障恢复。 | Nested retries can amplify load, cost and duplicate tool side effects.嵌套重试会放大负载、成本与重复工具副作用。 |
| Request transforms请求转换 | Inject approved defaults, metadata or compatibility mapping.注入批准的默认值、元数据或兼容映射。 | Hidden mutation can change safety, output, cost and debugging behavior.隐藏修改会改变安全、输出、成本与调试行为。 |
Security controls for browser, mobile, and server clients浏览器、移动端与服务端客户端的安全控制
Never place an OpenAI provider key in browser JavaScript, a mobile binary, a desktop package, a public repository, or a user-controlled configuration file. A proxy keeps that secret on trusted infrastructure, but the proxy endpoint must not become an unauthenticated relay that anyone can spend through.
不要把 OpenAI 供应商密钥放进浏览器 JavaScript、移动端安装包、桌面软件、公开代码仓库或用户可控配置文件。代理可以把密钥保留在受信基础设施中，但代理端点绝不能因此变成任何人都能消耗额度的匿名转发器。
Validate the user session or workload identity, then authorize the specific model alias, operation, tenant, and feature. Do not use one static proxy key for every client.
Allowlist paths, methods, models, tools, and parameters. Limit content length, file size, tool count, maximum output, concurrent streams, and request rate before contacting upstream.
Configure upstream hosts on the server. Reject client-supplied URLs, redirect following, internal address ranges, unexpected schemes, and arbitrary proxy headers.
Strip authorization and hop-by-hop headers, sanitize logs, return stable client errors, and keep secret values out of traces, crash reports, and support bundles.
验证用户会话或工作负载身份，再授权具体模型别名、操作、租户与产品功能。不要让所有客户端共用一把静态代理密钥。
对白名单路径、方法、模型、工具与参数放行，并在访问上游前限制正文长度、文件大小、工具数量、最大输出、并发流与请求速率。
上游主机必须由服务端配置。拒绝客户端提供的 URL、自动重定向、内部地址、异常协议与任意代理 Header。
移除 Authorization 与 Hop-by-Hop Header，清理日志，返回稳定客户端错误，并确保密钥值不进入调用链、崩溃报告或支持包。
Streaming is a protocol contract, not a response body流式传输是一套协议契约，不只是响应正文
A production proxy must preserve event order, framing, keep-alive behavior, backpressure, client cancellation, and the final completion state. Buffering the complete response before returning it destroys time-to-first-token and may increase memory use dramatically. Applying ordinary compression or response transformations can also delay events or break clients.
生产代理必须保持事件顺序、帧格式、连接保活、背压、客户端取消与最终结束状态。先缓冲完整响应再一次性返回，会失去首 Token 延迟优势，并可能显著增加内存占用。普通压缩或响应转换也可能延迟事件，甚至破坏客户端解析。
Use separate connection, first-byte, idle-stream, and total deadlines. When the client disconnects, propagate cancellation upstream when supported and stop consuming resources. If the stream ends after partial output, record the amount delivered and return or surface an explicit incomplete state; do not silently restart the request and concatenate a second model response.
连接、首字节、流空闲与总时长应分别设置超时。客户端断开后，在上游支持的情况下应传递取消并停止消耗资源。如果已经输出部分内容后流被中断，要记录已传输的数据量，并明确标记结果不完整；不能静默重启请求，再把第二次模型响应拼接上去。
Give retries one owner and a total deadline重试只能有一个责任方，并受总截止时间约束
The client SDK, application service, proxy, and upstream platform may each retry. When several layers retry independently, one user action can multiply into many paid requests and produce a traffic spike during an outage. Choose one layer to own retries, pass attempt metadata, and enforce a total deadline across connection, queueing, generation, and any retry delay.
客户端 SDK、应用服务、代理与上游平台都可能自带重试。如果多层各自重试，一次用户操作会被放大成多次付费请求，并在事故期间制造流量尖峰。应明确唯一重试层，传递尝试次数元数据，并让连接、排队、生成与重试等待共享一个总截止时间。
| Condition情况 | Safer behavior更安全的处理 | Reason原因 |
|---|---|---|
| Connection failed before any response建立连接前失败 | One bounded retry with jitter if deadline remains在总时间允许时进行一次带抖动的有限重试 | No output was delivered; transient recovery may help尚未交付输出，短暂故障可能恢复 |
| Upstream rate limit上游限流 | Honor retry guidance, queue only within a strict deadline, or return a stable overload error遵循重试提示，仅在严格时间内排队，否则返回稳定过载错误 | Immediate retry can worsen saturation and cost立即重试会加重饱和与成本 |
| Partial stream already delivered已经交付部分流 | Do not automatically retry; mark incomplete不要自动重试，明确标记不完整 | A second response may duplicate or contradict output第二次响应可能重复或冲突 |
| Tool call may have side effects工具调用可能产生副作用 | Require idempotency or explicit recovery要求幂等或明确恢复流程 | Blind retry can repeat external actions盲目重试会重复外部动作 |
Log enough to diagnose, not enough to leak日志要足以诊断，但不能多到造成泄露
Useful proxy telemetry connects the client request ID, authenticated actor, tenant, model alias, native upstream request ID, native model, attempt count, status, timing stages, token usage, finish reason, and transformation version. These fields usually answer availability and cost questions without storing full prompt or response bodies.
有用的代理遥测应关联客户端请求 ID、已认证主体、租户、模型别名、上游原生请求 ID、原生模型、尝试次数、状态、各阶段耗时、Token 用量、结束原因与转换版本。这些字段通常足以回答可用性与成本问题，而无需保存完整提示词和响应正文。
If payload capture is required for quality or support, make it a separate, access-controlled mode with explicit sampling, redaction, retention, customer consent, and deletion. Do not allow debug logging to become a permanent shadow database. Ensure third-party APM, exception tracking, and support systems follow the same data policy.
如果质量或支持确实需要采集正文，应把它设计成独立、受权限控制的模式，明确采样、脱敏、保留、客户同意与删除规则。不要让调试日志演变成永久的影子数据库。第三方 APM、异常跟踪与支持系统也必须遵守同一数据政策。
Evaluate the proxy as a security product把代理当作安全产品评估
Identify ownership, region, network path, updates, dependencies and support boundary.
Document encryption, logging, redaction, retention, training use and deletion.
Version every header, field, prompt, tool, stream, error and usage transformation.
Test direct or alternate access, credential rotation and proxy-unavailable behavior.
识别所有权、区域、网络路径、更新、依赖与支持边界。
记录加密、日志、脱敏、保留、训练使用与删除。
版本化每个 Header、字段、提示词、工具、流、错误与用量转换。
测试直连或备用访问、凭证轮换与代理不可用行为。
OpenAI API proxy production checklistOpenAI API 代理生产检查表
Every client is authenticated; operations are authorized; credentials are scoped; per-user or workload quotas exist; anomalous traffic and spend are detected.
Paths, methods, models, tools, MIME types, parameters, body size, output tokens, concurrency, and upstream endpoints are constrained server-side.
Payload exposure, redaction, sampling, log destinations, retention, deletion, operator access, subprocessors, and support workflows are documented and tested.
Direct and proxied results are compared for streaming events, cancellation, tool calls, structured output, errors, native IDs, usage, and finish states.
Connection, first-byte, idle, and total deadlines exist; one retry owner is defined; overload is bounded; partial streams and client disconnects are visible.
Key rotation, deployment rollback, proxy outage, upstream outage, telemetry failure, configuration recovery, incident export, and emergency alternate access are rehearsed.
所有客户端都经过认证；操作按权限授权；凭证范围受限；具备用户或工作负载级配额；能发现异常流量与支出。
路径、方法、模型、工具、MIME 类型、参数、正文大小、输出 Token、并发与上游端点均由服务端限制。
正文暴露、脱敏、采样、日志目的地、保留、删除、运维访问、次级处理方与支持流程都有文档并经过测试。
对比直连与代理的流式事件、取消、工具调用、结构化输出、错误、原生 ID、用量与结束状态。
分别设置连接、首字节、空闲与总超时；明确唯一重试方；限制过载；部分流与客户端断开均可见。
演练密钥轮换、发布回滚、代理故障、上游故障、遥测故障、配置恢复、事故证据导出与紧急备用访问。
Threat-model and failure-test the proxy为代理做威胁建模与故障测试
- Map trust boundaries, payload classes, credentials and every storage location.
- Diff proxied and direct requests, streams, errors, usage and native IDs.
- Inject proxy timeout, restart, partial stream, upstream 429 and credential failure.
- Verify retries cannot duplicate non-idempotent tool or external actions.
- Exercise emergency bypass, key rotation and incident evidence export.
- 映射信任边界、请求数据类别、凭证与每个存储位置。
- 对比代理与直连的请求、流、错误、用量与原生 ID。
- 注入代理超时、重启、部分流、上游 429 与凭证故障。
- 验证重试不能重复非幂等工具或外部动作。
- 演练紧急旁路、密钥轮换与事故证据导出。
Make the checkpoint explicit显式表达检查点
Clients authenticate to the proxy with scoped identities. The proxy authorizes the workload, validates the OpenAI-shaped request, applies versioned and observable transforms, enforces budgets and forwards with a protected upstream credential. The response retains native IDs and transformation evidence. Direct rollback is isolated and tested.
客户端用受限身份向代理认证；代理授权工作负载、验证 OpenAI 形状请求、应用版本化且可观测的转换、执行预算，并用受保护的上游凭证转发。响应保留原生 ID 与转换证据；直连回滚被隔离并经过测试。
Production rule: a proxy may simplify client access only after its additional trust and failure modes are accepted.
生产规则：只有接受代理新增的信任与故障模式后，才能用它简化客户端访问。
Common OpenAI API proxy failure modesOpenAI API 代理常见失败模式
The provider key is hidden, but anyone can call the public proxy and consume quota because caller identity and authorization were never implemented.
A client-controlled base URL, redirect, or header turns the proxy into an SSRF path toward internal services or attacker-controlled endpoints.
Authorization headers, prompts, files, or responses enter ordinary application logs, APM traces, exception reports, or support bundles.
The proxy waits for the full response, increasing time to first token, memory, timeout risk, and the impact of large generations.
Client, application, proxy, and upstream retries combine, multiplying paid requests exactly when the provider is overloaded.
Default prompts, tools, parameters, model aliases, or safety fields change without a versioned record, making quality and incidents impossible to reproduce.
供应商密钥虽然被隐藏，但代理没有实现调用方身份与授权，任何人都能通过公开端点消耗额度。
客户端可控的 Base URL、重定向或 Header 把代理变成 SSRF 通道，能够访问内部服务或攻击者端点。
Authorization Header、提示词、文件或响应进入普通应用日志、APM 调用链、异常报告或支持包。
代理等待完整响应后再返回，导致首 Token 更慢、内存更高、超时风险更大，并放大长文本生成的影响。
客户端、应用、代理与上游各自重试，在供应商最拥堵的时候把一次付费请求放大成多次。
默认提示词、工具、参数、模型别名或安全字段发生变化，却没有版本记录，导致质量和事故无法复现。
A model proxy is not an external capability catalog模型代理不是外部能力目录
The proxy governs OpenAI-shaped inference traffic. QVeris complements it by discovering and calling external APIs, tools, services and live data. Keep tool identities and side-effect evidence distinct from the model proxy, then connect both through the parent workflow trace.
代理治理 OpenAI 形状的推理流量；QVeris 补充外部 API、工具、服务与实时数据的发现和调用。让工具身份与副作用证据独立于模型代理，再通过父工作流调用链连接两者。
Keep the proxy policy smaller than the API让代理 Policy 小于 API 本身
A safe proxy should authenticate the caller, resolve an allowlisted model alias, enforce request limits, attach trace context, and forward to a configured upstream. It should not accept arbitrary upstream URLs, raw provider keys, or unbounded request bodies from clients.
安全代理应认证调用方、解析 Allowlist 中的模型别名、限制请求、附加调用链上下文并转发到已配置 Upstream。它不应接受客户端提供的任意 Upstream URL、原始供应商密钥或无限请求 Body。
POST /v1/chat/completions
authenticate(caller_token)
require(content_length <= 1_000_000)
alias = require_allowlisted(request.model)
policy = load_versioned_policy(caller, alias)
upstream = policy.endpoint # server-side configuration only
key = secret_store.get(policy.key_ref)
trace_id = new_trace_id()
response = forward(upstream, key, request, trace_id, timeout=30s)
log(trace_id, caller, alias, response.model, response.usage, response.status)
return redact_and_normalize(response)- Strip hop-by-hop headers and never log authorization or full sensitive prompts.
- Define exactly one retry owner and prevent duplicate non-idempotent tool effects.
- Test SSRF, redirect, oversized body, invalid JSON, timeout, and upstream error cases.
- 移除 Hop-by-Hop Header，绝不记录 Authorization 或完整敏感提示词。
- 明确唯一重试负责人，并防止非幂等工具 Effect 重复。
- 测试 SSRF、重定向、超大 Body、无效 JSON、超时与 Upstream Error。
Verified implementation reference: OWASP API Security.
实施参考已根据官方资料核验：OWASP API Security。
FAQ
No. It is an intermediary operated by you or a third party, with its own controls and risks.
To isolate keys, centralize policy, apply limits, add evidence or bridge a compatible client contract.
Request mutations, retry behavior, data handling, upstream identity, native IDs and final error causes.
不是。它是由你或第三方运营的中间层，拥有自己的控制与风险。
为了隔离密钥、集中策略、应用限制、增加证据或桥接兼容客户端契约。
请求修改、重试行为、数据处理、上游身份、原生 ID 与最终错误原因。
More OpenAI API proxy questions更多 OpenAI API 代理问题
Only when it authenticates users, authorizes operations, validates and limits input, blocks arbitrary upstream URLs, controls abuse, and never returns the provider key.
It does not have to, but it can see them in transit. Logging, redaction, retention, access, deletion, and third-party telemetry must be explicitly defined.
A proxy is the forwarding boundary. An AI gateway normally adds model-aware routing, normalization, budgets, guardrails, caching, and governance. One product may perform both roles.
只有在代理认证用户、授权操作、验证并限制输入、阻止任意上游 URL、控制滥用且永不返回供应商密钥时，才是安全方案。
代理不一定保存，但传输过程中能够看到提示词，因此必须明确日志、脱敏、保留、访问、删除与第三方遥测规则。
代理主要负责转发边界；AI 网关通常还增加模型感知路由、协议统一、预算、护栏、缓存与治理。同一产品也可能同时承担两种角色。
Official sources and further reading官方资料与延伸阅读
OpenAI APIs, supported fields, data controls, and SDK behavior evolve. Verify current primary documentation and test the exact endpoint and features your proxy forwards.
OpenAI API、支持字段、数据控制与 SDK 行为会持续变化。上线前应核对最新一手文档，并测试代理实际转发的具体端点与功能。