AutoGen + Custom LLM
Implement the Contract, Not a ShortcutAutoGen 自定义 LLM：实现正式契约，不走临时捷径
AutoGen accepts model clients through defined interfaces. A production integration must declare real capabilities, normalize behavior carefully, isolate tools and preserve native evidence.
AutoGen 通过定义明确的接口接收模型客户端。生产集成必须如实声明能力、谨慎标准化行为、隔离工具，并保留原生证据。
TL;DR
Use a supported client, an OpenAI-compatible endpoint, or implement the client protocol.
Metadata enables framework behavior; it does not create missing model features.
Adapt requests, streams, tools, usage and errors without hiding native identity.
Verify conversations, termination, tools, structured output and cancellation end to end.
使用受支持客户端、OpenAI 兼容端点，或实现客户端协议。
元数据会启用框架行为，但不会凭空创造模型能力。
适配请求、流、工具、用量与错误，同时保留原生身份。
端到端验证对话、终止、工具、结构化输出与取消。
Three integration paths三种集成路径
Current AutoGen separates AgentChat and Core from provider implementations in autogen-ext. Use a maintained extension such as an existing ChatCompletionClient when it meets the workload. This preserves the framework's expected message, usage, cancellation and component behavior instead of creating a parallel integration surface.
当前 AutoGen 将 AgentChat、Core 与 autogen-ext 中的供应商实现分离。若现有 ChatCompletionClient 能满足工作负载，应优先使用维护中的 Extension，以保留框架预期的 Message、Usage、Cancellation 与 Component 行为，而不是另建一套平行集成。
For an OpenAI-compatible Chat Completions service, OpenAIChatCompletionClient can be configured with a custom base URL. AutoGen's documentation cautions that compatible endpoints may still differ and are not automatically guaranteed. Supply accurate model_info—including vision, function calling, JSON and structured-output support—when the client cannot infer capabilities.
对于兼容 OpenAI Chat Completions 的服务，可以为 OpenAIChatCompletionClient 配置自定义 Base URL。AutoGen 官方文档也提醒，兼容端点仍可能存在细微差异，不能自动视为受保证实现。当 Client 无法推断能力时，应准确提供 model_info，包括 Vision、Function Calling、JSON 与 Structured Output 支持。
For a provider with a different protocol or lifecycle, implement the AutoGen ChatCompletionClient contract or a dedicated extension. Do not force incompatible streaming, tool, usage or error semantics through a thin base-URL swap. A narrow explicit adapter is easier to test and upgrade than a compatibility layer filled with hidden exceptions.
对于协议或生命周期不同的供应商，应实现 AutoGen ChatCompletionClient 契约或专用 Extension。不要只更换 Base URL 就强行掩盖流式、工具、用量或错误语义差异。窄而明确的 Adapter 比充满隐藏例外的兼容层更容易测试和升级。
Integration path comparison集成路径对比
| Path路径 | Best fit最适合 | Verify before choosing选择前验证 |
|---|---|---|
| Supported extension受支持 Extension | The provider is covered and required features are documented.供应商已覆盖且所需功能有文档。 | Pin compatible package versions and run workload contract tests.固定兼容包版本，并运行负载契约测试。 |
| OpenAI-compatible clientOpenAI 兼容客户端 | Endpoint behavior closely matches required API semantics.端点行为接近所需 API 语义。 | Test tools, streams, finish reasons, usage, errors and cancellation.测试工具、流、结束原因、用量、错误与取消。 |
| Custom client自定义客户端 | Protocol or behaviors require deliberate translation.协议或行为需要明确转换。 | Implement lifecycle, async behavior, usage, capabilities and error contracts.实现生命周期、异步行为、用量、能力与错误契约。 |
| Gateway adapter网关适配 | Several providers need one audited policy boundary.多个供应商需要统一审计策略边界。 | Keep provider evidence and avoid claiming unsupported feature parity.保留供应商证据，不宣称不存在的功能等价。 |
Production integration checklist生产集成检查清单
Verify operator, TLS, region, retention and credential scope.
Declare vision, function calling, JSON or structured output accurately.
Close clients, honor cancellation, bound timeouts and avoid nested retries.
Keep provider IDs, usage, finish reasons and raw error context for support.
验证运营方、TLS、区域、保留策略与凭证范围。
准确声明视觉、函数调用、JSON 或结构化输出能力。
关闭客户端、尊重取消、限制超时并避免嵌套重试。
保留供应商 ID、用量、结束原因与原始错误上下文。
Contract-test the model client对模型客户端做契约测试
Test the client directly first, then inside the real AgentChat workflow. Direct tests isolate protocol mapping; workflow tests reveal prompt growth, handoffs, repeated tool calls and termination behavior that a one-shot request cannot expose.
应先直接测试 Model Client，再放入真实 AgentChat 工作流中测试。直接测试用于隔离协议映射问题；工作流测试则能暴露一次请求看不到的 Prompt 增长、Handoff、重复工具调用与终止行为。
- Test simple chat, multi-turn context, streaming event order, cancellation and client close behavior.
- Verify tool schemas, parallel calls, result correlation, malformed arguments and unsupported capability rejection.
- Exercise structured output, safety refusal, context and output limits, usage reporting and provider errors.
- Confirm declared
model_infomatches observed behavior instead of enabling features optimistically. - Run AgentChat golden tasks and compare complete workflow outcomes, attempts, latency and cost.
- 测试简单对话、多轮上下文、流式事件顺序、取消以及 Client Close 行为。
- 验证工具 Schema、并行调用、结果关联、异常参数和不支持能力的拒绝。
- 覆盖结构化输出、安全拒绝、上下文与输出限制、Usage Reporting 和供应商错误。
- 确认声明的
model_info与观察行为一致，而不是乐观地开启功能。 - 运行 AgentChat Golden Task，并比较完整工作流结果、尝试次数、延迟与成本。
Place a narrow adapter at the trust boundary在信任边界放置窄适配器
AutoGen AgentChat calls a model-client interface. The client or gateway should translate only necessary protocol differences, enforce scoped credentials and policy, and return normalized results plus provider-native evidence. Keep the adapter stateless where possible; conversation context and workflow state belong to the agent layer rather than a hidden proxy session.
AutoGen AgentChat 调用 Model Client Interface。Client 或 Gateway 只应转换必要协议差异、执行受限凭证与策略，并返回标准化结果和供应商原生证据。Adapter 应尽量保持无状态；会话上下文与工作流状态属于 Agent 层，不应藏在 Proxy Session 中。
Version the client configuration, model information and adapter together. Every trace should retain that version, the native request ID, usage and finish reason. Tool execution, code sandboxes and external side effects remain separate services with independent approval and idempotency controls, even when the model client emits their call arguments.
Client 配置、Model Info 与 Adapter 应一起版本化。每条 Trace 都要保留该版本、原生 Request ID、Usage 与 Finish Reason。即使 Model Client 生成了调用参数，工具执行、代码 Sandbox 与外部副作用仍应作为独立服务，并拥有单独审批与幂等控制。
Production rule: never mark a capability true until the target endpoint passes that capability's contract test.
生产规则：目标端点未通过对应能力的契约测试前，绝不能把该能力标记为 true。
Keep external capabilities outside the model adapter将外部能力放在模型适配器之外
AutoGen and its model clients manage agent conversations and inference. QVeris adds governed Discover → Inspect → Call access to external APIs, tools, services and live data. Inspect a capability's schema, permissions and provider evidence before presenting it to an agent, rather than assuming every custom model handles every tool shape correctly.
AutoGen 与 Model Client 管理 Agent 对话和推理；QVeris 通过 Discover → Inspect → Call 补充外部 API、工具、服务与实时数据。在把能力提供给 Agent 前，应检查其 Schema、权限与 Provider Evidence，而不是假设每个自定义模型都能正确处理所有工具结构。
Keep capability credentials, execution evidence, retry policy and idempotency separate from the model adapter, then link both paths with a parent trace. If a model client fails after a tool action completes, resume from the verified action state instead of replaying the entire AgentChat turn and duplicating the side effect.
能力凭证、执行证据、重试策略和幂等控制应与 Model Adapter 分离，再通过父 Trace 关联两条路径。如果工具动作完成后 Model Client 才失败，应从已验证动作状态继续，而不是重放整个 AgentChat Turn 并重复副作用。
Instantiate one AutoGen model client explicitly显式创建一个 AutoGen 模型 Client
Current AutoGen AgentChat uses model client objects. For an OpenAI-compatible endpoint, configure the exact model, base URL, key, and model_info instead of relying on a broad environment override. Close the client so pooled connections are released cleanly.
当前 AutoGen AgentChat 使用模型 Client 对象。连接 OpenAI 兼容端点时，应显式配置准确模型、基础地址（Base URL）、密钥与 model_info，而不是依赖宽泛的环境覆盖；完成后关闭客户端，以正确释放连接池。
from autogen_core.models import UserMessage
from autogen_ext.models.openai import OpenAIChatCompletionClient
import os, asyncio
async def main():
client = OpenAIChatCompletionClient(
model=os.environ["LLM_MODEL_ID"],
base_url=os.environ["LLM_GATEWAY_BASE_URL"],
api_key=os.environ["LLM_GATEWAY_API_KEY"],
model_info={"vision": False, "function_calling": True,
"json_output": True, "family": "unknown"},
)
result = await client.create([UserMessage(content="Reply AUTOGEN_OK", source="user")])
print(result.content, result.usage)
await client.close()
asyncio.run(main())- Set model_info from verified capabilities, not marketing claims.
- Enable AutoGen event logging and correlate it with the gateway request ID.
- Test tool schemas and parallel calls before enabling multi-agent concurrency.
- model_info 必须来自已验证能力，而不是营销描述。
- 启用 AutoGen Event Logging，并与网关请求 ID 关联。
- 启用多智能体并发前先测试工具结构定义与并行调用。
Verified implementation reference: AutoGen model clients.
实施参考已根据官方资料核验：AutoGen model clients。
FAQ
Yes, but AutoGen notes that compatible services can differ; test required semantics.
It describes capabilities used by the framework; it does not alter the model.
In an isolated executor with resource, filesystem and network controls.
可以，但 AutoGen 明确提示兼容服务可能有差异，必须测试所需语义。
它描述框架使用的能力，不会改变模型本身。
在具备资源、文件系统与网络控制的隔离 Executor 中运行。