Track LLM Usage & Cost
One Trace, Complete Evidence
追踪 LLM 用量与成本：用一条调用链串起完整证据
A provider bill is not enough for product decisions, and an application token counter is not enough for accounting. Join both with request-level evidence and reconciliation.
供应商账单不足以支持产品决策，应用侧 Token 计数也不足以完成核算。应使用请求级证据连接两者并持续对账。
TL;DR
Link app trace, gateway request, provider attempt, response and tool-call IDs.
Record success, retry and fallback usage rather than only the final response.
Map units for analysis while retaining native fields and billing dimensions.
Compare internal ledgers with provider exports and explain every material difference.
连接 App 调用链、网关请求、供应商 Attempt、Response 与工具调用 ID。
记录成功、重试与回退用量，而不是只记录最终响应。
为分析映射单位，同时保留原生字段与计费维度。
比较内部账本与供应商导出，并解释每个重要差异。
The evidence model 证据模型
For every provider attempt, store tenant, project, workload, route, provider, model version, native request ID, input and output units, cache dimensions, retries, latency, status and allocation tags.
为每次供应商尝试存储租户、Project、工作负载、路由、供应商、模型 Version、原生请求 ID、输入输出单位、缓存维度、重试、延迟、状态与分摊标签。
Treat tool calls, media processing, grounding, storage, batch and priority modes as separate billable dimensions when the provider exposes them. Do not compress them into a guessed token total.
当供应商公开工具调用、媒体处理、Grounding、存储、批处理或 Priority 等单独计费维度时，应分别记录，不要压缩为猜测的 Token 总数。
Usage data sources 用量数据来源
| Source 来源 | Best fit 最适合 | Verify before choosing 选择前验证 |
|---|---|---|
| Response usage 响应 Usage | Low-latency product analytics and per-request evidence. 低延迟产品分析与请求级证据。 | Missing responses, streams, retries and provider-specific fields. 检查缺失响应、流、重试与供应商特定字段。 |
| Gateway ledger 网关账本 | Cross-provider normalization and tenant allocation. 跨供应商标准化与租户分摊。 | Schema versioning, duplicate events and retry ownership. 检查结构定义版本、重复事件与重试归属。 |
| Provider usage export 供应商用量导出 | Authoritative provider-side aggregation. 权威供应商侧聚合。 | Time zones, delayed data, model aliases and billing cutoffs. 检查时区、延迟数据、模型别名与计费截止。 |
| Invoice or contract 账单或合同 | Financial settlement and audit. 财务结算与审计。 | Credits, taxes, commitments, special tiers and non-token charges. 检查抵扣、税费、承诺、特殊层级与非 Token 费用。 |
Ledger design checklist 账本设计清单
Append time-ordered usage evidence; correct with adjustment events.
Require tenant, environment, team, product and workload ownership.
Store native unit name, value, source and normalization version.
Avoid logging prompts or outputs when usage metadata is sufficient.
追加按时间排序的用量证据，用调整事件纠错。
要求租户、Environment、Team、Product 与工作负载归属。
存储原生单位名、数值、来源与标准化版本。
用量元数据足够时，不记录提示词或输出内容。
Build a reconciliation loop 建立对账循环
- Sample traces and verify every hop and attempt has a stable identifier.
- Replay retries, fallbacks, cache hits, streams and failed responses.
- Compare daily ledger totals with provider usage exports by model and project.
- Reconcile billing periods and document matched, explained and unexplained deltas.
- 采样调用链，验证每个 Hop 与 Attempt 都有稳定标识。
- 回放重试、回退、缓存命中、流与失败响应。
- 按模型与项目比较每日账本总量与供应商用量导出。
- 按账期对账，并记录已匹配、已解释与未解释差异。
Join operational traces with a finance ledger 连接运营调用链与财务账本
The application emits a trace ID. The gateway creates a request ID and one child attempt per provider call. Usage events flow into an immutable ledger. A reconciliation job aligns provider exports and invoices, then attaches adjustments and evidence instead of rewriting history.
应用生成调用链 ID；网关生成请求 ID，并为每次供应商调用创建子 Attempt；Usage Event 进入不可变账本；对账任务将供应商导出与账单对齐，并追加调整与证据，而不是重写历史。
Production rule: never report model cost without counting retries, fallbacks and non-token billable dimensions.
生产规则：未计入重试、回退与非 Token 计费维度时，不得报告模型成本。
Trace tool and live-data charges separately 单独追踪工具与实时数据费用
QVeris calls external APIs, tools, services and live data. Propagate the parent trace but record capability usage, provider evidence and cost in a separate tool ledger so inference and external actions remain explainable.
QVeris 调用外部 API、工具、服务与实时数据。应传播父调用链，但在独立工具账本中记录能力用量、供应商证据与成本，使推理和外部动作都可解释。
Calculate cost from a versioned price catalog 使用版本化 Price Catalog 计算成本
Store raw usage and the price version used for calculation. Recomputing historical events with today's prices destroys auditability, while storing only a provider-reported dollar total makes allocation and anomaly review impossible.
同时保存原始用量与计算所使用的 Price Version。用今天价格重算历史 Event 会破坏审计能力，而只保存供应商返回的美元总额又无法进行分摊与异常复核。
from decimal import Decimal
def request_cost(usage, price):
million = Decimal("1000000")
uncached = Decimal(usage["input_tokens"] - usage.get("cached_input_tokens", 0))
cached = Decimal(usage.get("cached_input_tokens", 0))
output = Decimal(usage["output_tokens"])
return {
"input_usd": uncached / million * Decimal(price["input_per_mtok"]),
"cached_input_usd": cached / million * Decimal(price.get("cached_input_per_mtok", 0)),
"output_usd": output / million * Decimal(price["output_per_mtok"]),
"price_version": price["version"],
"currency": "USD",
}
# Persist the original usage event and each calculated component.
- Map native model IDs to prices with effective-from and effective-to timestamps.
- Keep provider fees, gateway fees, tools, storage, taxes, credits, and discounts as separate components.
- Reconcile daily ledger totals to provider exports before using the data for chargeback.
- 用 Effective-From 与 Effective-To 时间把原生模型 ID 映射到价格。
- 把供应商费、网关费、工具、存储、税费、Credits 与折扣作为独立组件。
- 用于内部计费前，每日将账本汇总与供应商导出对账。
Verified implementation reference: OpenTelemetry GenAI conventions.
实施参考已根据官方资料核验：OpenTelemetry GenAI conventions。
Build an Effective Cost Model for LLM usage and cost tracking为LLM 用量与成本追踪建立有效成本模型
When evaluating Track LLM Usage Cost, list price is an input, not the decision. Compare the cost of an accepted production outcome after quality, retries, latency, operational work, and non-token charges are included.
评估“追踪 LLM 用量与成本”时，目录价只是输入，而不是最终决策。应在计入质量、重试、延迟、运营工作和非 Token 费用后，比较获得一个合格生产结果的成本。
Sample production-shaped tasks and record provider request IDs, model versions, token classes, cached input, reasoning usage, tool charges, retries, user and tenant attribution, and price-effective dates. Use percentiles and task classes rather than one average prompt so long-context and output-heavy requests remain visible.
抽取接近生产形态的任务，并记录供应商请求 ID、模型版本、Token 类别、缓存输入、推理用量、工具费用、重试、用户与租户归因和价格生效日期。使用分位数和任务类别，而不是单个平均提示词，确保长上下文与高输出请求不会被隐藏。
When evaluating Track LLM Usage Cost, measure completion, rubric score, structured-output validity, tool accuracy, retry rate, and human-review time. Divide total run cost by accepted outcomes, not by raw requests.
评估“追踪 LLM 用量与成本”时，对每个模型与路由衡量完成率、评分标准得分、结构化输出有效率、工具准确率、重试率和人工复核时间，并用总运行成本除以合格结果，而不是原始请求数。
When evaluating Track LLM Usage Cost, add gateway or aggregator fees, storage, egress, cache writes, evaluations, observability, support, engineering, incident handling, reserved capacity, unused credits, taxes, and the business cost of latency or failed work.
评估“追踪 LLM 用量与成本”时，加入网关或聚合费用、存储、流量、缓存写入、评估、可观察性、支持、工程、事故处理、预留容量、未用额度、税费，以及延迟或任务失败的业务成本。
When evaluating Track LLM Usage Cost, store source URL, retrieval date, currency, region, service tier, thresholds, discounts, and model version. Recompute scenarios when a catalog changes and alert when observed invoice cost diverges from the estimate.
评估“追踪 LLM 用量与成本”时，保存来源链接、获取日期、币种、区域、服务层级、阈值、折扣和模型版本。目录变化时重新计算情景，并在实际发票成本偏离估算时发出告警。
FAQ
Use them for estimates; reconcile billing with provider-native usage and invoices.
Attribute every attempt to the parent workload and mark its final disposition.
Only when justified and governed; usage tracking usually needs metadata, not content.
可用于估算；计费应与供应商原生用量和账单对账。
把每次尝试归属到父工作负载，并标记最终处置。
仅在有正当理由且受治理时；用量追踪通常只需元数据。