Martian LLM Router Alternatives for Current Production Use Martian LLM 路由器替代方案：面向当前生产环境
Treat the historical Martian router as a routing idea, not a confirmed current SKU. Rebuild the requirement around task quality, cost, latency, availability, and operator ownership.
把历史 Martian 路由器视为一种路由思路，而不是已确认的当前 SKU。围绕任务质量、成本、延迟、可用性和运营责任重新定义需求。
TL;DR
The historical Martian model-router URL returns no current product page, while the current site emphasizes interpretability and research.
A quality-aware router must be evaluated on your prompts, graders, models, latency budget, cost ceiling, and failure policy.
Optimization-focused routers choose a model by predicted fit; gateways emphasize provider access, fallback, policy, and operations.
Official documentation, pricing, terms, status, support, and a contract must agree before production selection.
历史 Martian 模型路由器 URL 不再提供当前产品页，而现官网强调可解释性与研究。
质量感知路由器必须使用你的提示词、评分器、模型、延迟预算、成本上限与故障策略评估。
优化型路由器按预测适配选择模型；网关更强调供应商访问、回退、策略与运营。
官方文档、定价、条款、状态、支持与合同必须一致，才能用于生产选型。
Product status is part of technical due diligence 产品状态也是技术尽调
A dead product URL is not proof that a company disappeared, but it is enough to stop copying an old feature matrix. At the time of this review, Martian's public site focuses on understanding and interpreting models, and the historical router page is unavailable. Ask for a current product brief, documentation root, pricing, terms, status page, support commitment, and architecture before treating any router offer as active.
产品 URL 失效并不能证明公司消失，但足以停止复制旧功能表。本页审阅时，Martian 公开官网聚焦理解与解释模型，历史路由器页面不可用。在把任何路由器方案视为现役产品前，应要求提供当前产品说明、文档根目录、定价、条款、状态页、支持承诺与架构。
Date the evidence. Product direction can change after publication. Re-check the official site and obtain written confirmation for procurement.
给证据标注日期。产品方向可能在页面发布后变化；采购前应重新核对官网并获得书面确认。
Separate model optimization from gateway operations 区分模型优化与网关运营
Predicts which model is likely to meet a quality bar for a specific request and may trade quality against cost or latency.
Normalizes APIs, credentials, retries, fallback, load balancing, budgets, caching, telemetry, and policy.
Decides which tenant may use which model, in which region, under which budget, guardrail, and approval.
Maintains datasets, graders, regressions, and evidence that the routing policy still works as models change.
预测哪个模型最可能满足特定请求的质量门槛，并在质量、成本或延迟之间权衡。
标准化 API、凭证、重试、回退、负载均衡、预算、缓存、遥测与策略。
决定哪个租户可在哪个区域、预算、护栏与审批下使用哪个模型。
维护数据集、评分器、回归和证据，证明模型变化后路由策略仍有效。
Eight current alternatives by routing approach 按路由方式划分的 8 个当前替代方案
| Option 选项 | Routing approach 路由方式 | Proof required 所需证据 |
|---|---|---|
| Not Diamond | Model selection optimized for task performance 面向任务表现优化的模型选择 | Lift on your evaluation set 在自有评估集上的提升 |
| OpenRouter | Broad model access and provider routing 广模型访问与供应商路由 | Routing controls and provider economics 路由控制与供应商经济性 |
| Portkey | Policy-driven managed gateway 策略驱动的托管网关 | Governance and deployment fit 治理与部署匹配 |
| Requesty | Hosted routing, fallback, and caching 托管路由、回退与缓存 | Failure behavior and total cost 故障行为与总成本 |
| Vercel AI Gateway | Application-platform routing and fallbacks 应用平台路由与回退 | Platform affinity and portability 平台亲和与可迁移性 |
| LiteLLM | Self-hosted rules and provider abstraction 自托管规则与供应商抽象 | Operator cost and reliability 运维成本与可靠性 |
| Bifrost | High-throughput self-hosted data plane 高吞吐自托管数据面 | Control-plane completeness 控制面完整性 |
| RouteLLM | Open routing research and implementation patterns 开放路由研究与实现模式 | Production hardening and model freshness 生产强化与模型新鲜度 |
Build a router benchmark that resists gaming 建立不易被“刷分”的路由基准
Use a hidden, time-sliced evaluation set drawn from real traffic. Label hard constraints first: data region, model allowlist, context size, modality, tool support, and maximum latency. Then score task success, calibrated confidence, cost, first-token latency, total latency, and fallback behavior. Compare against simple baselines such as always-premium, always-cheap, and deterministic rules.
使用来自真实流量、隐藏且按时间切片的评估集。先标注数据区域、模型白名单、上下文长度、模态、工具支持和最大延迟等硬约束，再评分任务成功、置信度校准、成本、首 Token 延迟、总延迟与回退行为。必须与“始终使用高端模型”“始终使用低价模型”和确定性规则等简单基线比较。
- Report quality at a fixed cost and cost at a fixed quality, not one blended score.
- Re-run after provider model updates, pricing changes, and material prompt-distribution shifts.
- Include abstention: the router should admit when it cannot confidently choose.
- Log the reason, candidate set, chosen model, policy version, and realized outcome for every decision.
- 分别报告固定成本下的质量和固定质量下的成本，不用一个混合分数掩盖取舍。
- 供应商模型更新、价格变化或提示词分布显著改变后重新测试。
- 包含弃权机制：路由器无法可靠选择时应明确承认。
- 每次决策记录原因、候选集、所选模型、策略版本与实际结果。
Migrate the decision policy, not just the endpoint 迁移决策策略，而不只是端点
Export model candidates, constraints, quality thresholds, cost weights, latency ceilings, fallbacks, evaluation datasets, graders, and decision logs. Reproduce historical decisions offline with the candidate. Shadow live traffic, then canary a bounded segment. Keep the old path until both routing outcomes and gateway reliability reconcile.
导出模型候选、约束、质量阈值、成本权重、延迟上限、回退、评估集、评分器与决策日志。先用候选方案离线复现历史决策，再影子运行实时流量，最后 Canary 有界流量。在路由结果和网关可靠性都核对前，保留旧路径。
Route the model, then govern the action 先路由模型，再治理动作
A model router decides which model handles inference. QVeris helps the agent discover and call an external data source, API, or tool after that inference. Join the router decision and capability call with one trace ID so quality, cost, and downstream consequences can be investigated together.
模型路由器决定由哪个模型处理推理；QVeris 帮助智能体在推理后发现并调用外部数据源、API 或工具。用同一个调用链 ID 连接路由决策与能力调用，以便一起调查质量、成本与下游影响。
A Production Evaluation Plan for Martian LLM Router alternativesMartian LLM Router 替代方案的生产评估方案
A feature table can identify candidates, but it cannot prove operational fit. Evaluate Martian LLM Router alternatives with the workloads, policies, failure conditions, and evidence requirements that the team will actually own after migration.
功能表可以帮助筛选候选方案，却无法证明生产适配性。评估Martian LLM Router 替代方案时，应使用团队迁移后真正需要承担的工作负载、策略、失败条件和证据要求。
Inventory representative requests and record routing objectives, quality prediction, model eligibility, explainability, evaluation data, fallback behavior, and control-plane ownership. Include volumes, tail latency, quality thresholds, regulated data, operator steps, monthly spend, and the incidents the current system already knows how to handle.
盘点有代表性的请求，并记录路由目标、质量预测、模型资格、可解释性、评估数据、故障切换行为和控制面责任。同时纳入流量、长尾延迟、质量门槛、受监管数据、人工步骤、月度支出，以及现有系统已经能够处理的事故类型。
To validate Martian LLM Router Alternatives, replay saved cases against each candidate. Compare accepted parameters, streaming events, structured output, tool calls, error classes, usage accounting, and source metadata. Mark every difference as required, adaptable, or a migration blocker.
验证“Martian LLM 路由器替代方案”时，用保存的案例重放每个候选方案，比较参数、流式事件、结构化输出、工具调用、错误类别、用量计量和来源元数据，并将差异标记为必须保留、可以适配或阻断迁移。
To validate Martian LLM Router Alternatives, measure end-to-end task completion, output quality, p50 and tail latency, availability, retry amplification, fallback behavior, and accepted-result cost. Include rate limits, malformed responses, regional loss, schema drift, and provider outages.
验证“Martian LLM 路由器替代方案”时，衡量端到端任务完成率、输出质量、常规与长尾延迟、可用性、重试放大、故障切换行为和合格结果成本，并加入限流、畸形响应、区域丢失、Schema 漂移与供应商中断。
Before rolling out Martian LLM Router Alternatives, version routing and policy outside the vendor, preserve trace identifiers, stage read-only traffic first, define rollback signals, and retain a direct-provider or previous-platform path until evidence meets the acceptance threshold.
上线“Martian LLM 路由器替代方案”前，在供应商之外版本化路由与策略，保留追踪标识，先迁移只读流量，定义回滚信号，并在证据达到验收门槛前保留直连供应商或原平台路径。
FAQ
The former product URL is not active and the current site emphasizes interpretability research. Obtain direct, dated confirmation before relying on availability.
Not Diamond and open routing approaches such as RouteLLM are relevant comparisons; validate them against simple baselines on your tasks.
A gateway can execute deterministic policy and fallback, but it may not predict task quality. These can be separate layers.
No. QVeris focuses on governed discovery and invocation of external capabilities.
原产品 URL 已不可用，当前官网强调可解释性研究。依赖其可用性前应获得直接且带日期的确认。
Not Diamond 与 RouteLLM 等开放路由方法值得比较；必须在自有任务上与简单基线验证。
网关可执行确定性策略与回退，但未必预测任务质量；两者可以是独立层。
不会。QVeris 聚焦受治理的外部能力发现与调用。