Track LLM Usage & Cost
One Trace, Complete Evidence追踪 LLM 用量与成本：用一条调用链串起完整证据
A provider bill is not enough for product decisions, and an application token counter is not enough for accounting. Join both with request-level evidence and reconciliation.
供应商账单不足以支持产品决策，应用侧 Token 计数也不足以完成核算。应使用请求级证据连接两者并持续对账。
TL;DR
Link app trace, gateway request, provider attempt, response and tool-call IDs.
Record success, retry and fallback usage rather than only the final response.
Map units for analysis while retaining native fields and billing dimensions.
Compare internal ledgers with provider exports and explain every material difference.
连接 App 调用链、网关请求、供应商 Attempt、Response 与工具调用 ID。
记录成功、重试与回退用量，而不是只记录最终响应。
为分析映射单位，同时保留原生字段与计费维度。
比较内部账本与供应商导出，并解释每个重要差异。
The evidence model证据模型
Track usage at provider-attempt level, then roll it up to the user request and business workload. For every attempt, store tenant, project, environment, workload, route, provider, immutable model version, native request ID, input and output units, cache dimensions, retries, latency, status and allocation tags. Keep both provider-native values and normalized fields so later reporting does not erase how the source actually measured usage.
用量应先在 Provider Attempt 层记录，再汇总到用户请求与业务工作负载。每次尝试都应存储租户、Project、环境、工作负载、路由、供应商、不可变模型版本、原生 Request ID、输入输出单位、缓存维度、重试、延迟、状态和分摊标签。同时保留供应商原生值与标准化字段，避免后续报表抹去源端真正的计量方式。
A usage event is not yet a cost. Join it to a versioned price record with currency, unit, effective time and pricing source. This matters when cached input, reasoning units, long-context tiers or priority processing have different rates. Never recalculate old events using today's price without preserving the historical rate used for the original estimate.
Usage Event 还不等于成本。需要把它与带有币种、单位、生效时间和价格来源的版本化 Price Record 关联；缓存输入、Reasoning Unit、长上下文阶梯或 Priority Processing 可能采用不同费率。不能直接用今天的价格重算历史事件，而不保留当时估算所用的历史费率。
Treat tool calls, media processing, grounding, storage, batch and priority modes as separate billable dimensions when the provider exposes them. Do not compress them into a guessed token total. When a response omits usage—common for some failed or interrupted requests—mark it as unknown and reconcile from provider exports rather than silently recording zero.
当供应商公开工具调用、媒体处理、Grounding、存储、Batch 或 Priority 等独立计费维度时，应分别记录，不要压缩为猜测的 Token 总数。若失败或中断响应没有返回用量，应标记为 Unknown，并通过供应商导出对账，而不是静默记为零。
Usage data sources用量数据来源
| Source来源 | Best fit最适合 | Verify before choosing选择前验证 |
|---|---|---|
| Response usage响应 Usage | Low-latency product analytics and per-request evidence.低延迟产品分析与请求级证据。 | Missing responses, streams, retries and provider-specific fields.检查缺失响应、流、重试与供应商特定字段。 |
| Gateway ledger网关账本 | Cross-provider normalization and tenant allocation.跨供应商标准化与租户分摊。 | Schema versioning, duplicate events and retry ownership.检查结构定义版本、重复事件与重试归属。 |
| Provider usage export供应商用量导出 | Authoritative provider-side aggregation.权威供应商侧聚合。 | Time zones, delayed data, model aliases and billing cutoffs.检查时区、延迟数据、模型别名与计费截止。 |
| Invoice or contract账单或合同 | Financial settlement and audit.财务结算与审计。 | Credits, taxes, commitments, special tiers and non-token charges.检查抵扣、税费、承诺、特殊层级与非 Token 费用。 |
Ledger design checklist账本设计清单
Append time-ordered usage evidence; correct with adjustment events.
Require tenant, environment, team, product and workload ownership.
Store native unit name, value, source and normalization version.
Avoid logging prompts or outputs when usage metadata is sufficient.
追加按时间排序的用量证据，用调整事件纠错。
要求租户、Environment、Team、Product 与工作负载归属。
存储原生单位名、数值、来源与标准化版本。
用量元数据足够时，不记录提示词或输出内容。
Build a reconciliation loop建立对账循环
Operational telemetry gives fast estimates; provider exports and invoices provide later financial evidence. Reconciliation connects the two. Define tolerances by provider and dimension, because rounding, delayed usage, currency conversion and invoice adjustments can create legitimate differences.
运营遥测提供快速估算，供应商导出和账单则在之后提供财务证据。对账负责连接两者。应按供应商和计费维度定义容差，因为舍入、延迟用量、币种转换与账单调整都可能产生合理差异。
- Sample traces and verify every request, hop, attempt and billable action has a stable identifier.
- Replay retries, fallbacks, cache hits, long streams, cancellation and failed responses.
- Compare daily ledger totals with provider usage exports by model, project, region and billing dimension.
- Investigate missing usage, duplicate ingestion, late events and price-version mismatches before closing a period.
- Reconcile invoices and document matched, explained and unexplained deltas with owner and resolution status.
- 采样 Trace，验证每个请求、Hop、Attempt 与计费动作都有稳定标识。
- 回放重试、故障切换、缓存命中、长流、取消与失败响应。
- 按模型、Project、区域与计费维度比较每日账本和供应商用量导出。
- 账期关闭前调查缺失用量、重复摄取、延迟事件和价格版本不一致。
- 核对账单，并为已匹配、已解释与未解释差异记录负责人和处理状态。
Join operational traces with a finance ledger连接运营调用链与财务账本
The application emits a parent trace ID and business allocation tags. The gateway creates a request ID and one child attempt for every provider call, including retries and fallbacks. Usage events enter an append-only ledger with a deterministic event key, source timestamp and ingestion timestamp so delayed or repeated delivery can be handled without double counting.
应用生成父 Trace ID 与业务分摊标签；网关为每次 Provider Call（包括重试和故障切换）创建 Request ID 与子 Attempt。Usage Event 进入只追加账本，并携带确定性 Event Key、源时间与摄取时间，以便处理延迟或重复投递而不重复计数。
A pricing service resolves the rate effective at the event time and produces an estimate linked to the original usage record. A reconciliation job later aligns provider exports and invoices, then appends adjustments and supporting evidence instead of rewriting history. Dashboards can show request cost quickly while finance retains an auditable path from invoice line to workload.
Pricing Service 根据事件发生时间解析当时生效的费率，生成与原始用量记录关联的估算。对账任务随后将供应商导出与账单对齐，通过追加 Adjustment 与支持证据处理差异，而不是重写历史。这样 Dashboard 可以快速展示请求成本，财务也能从账单行追溯到具体工作负载。
Production rule: never report complete model cost without counting retries, fallbacks, unknown usage and non-token billable dimensions.
生产规则：未计入重试、故障切换、未知用量与非 Token 计费维度时，不得把结果称为完整模型成本。
Trace tool and live-data charges separately单独追踪工具与实时数据费用
QVeris calls external APIs, tools, services and live data whose commercial units may be requests, records, seats, data packages or provider-specific credits rather than tokens. Propagate the parent trace, tenant and workload tags, but record capability ID, external provider evidence, unit, quantity and price version in a separate tool ledger.
QVeris 调用外部 API、工具、服务与实时数据，其商业计量单位可能是请求、记录、席位、数据包或供应商专用 Credit，而不是 Token。应传播父 Trace、租户和工作负载标签，同时在独立工具账本中记录 Capability ID、外部供应商证据、单位、数量和价格版本。
Join model and tool ledgers only for workflow reporting. Keeping the source records separate prevents an expensive data lookup from being mislabeled as model spend and makes retries visible: one model retry may reuse an existing tool result, while another workflow may legitimately refresh it because the data freshness window expired.
模型账本与工具账本只在工作流报表层汇总。源记录保持独立，可避免昂贵的数据查询被误标为模型成本，也能清楚显示重试行为：一次模型重试可能复用已有工具结果，而另一条工作流则可能因新鲜度窗口过期而合理刷新。
Calculate cost from a versioned price catalog使用版本化 Price Catalog 计算成本
Store raw usage and the price version used for calculation. Recomputing historical events with today's prices destroys auditability, while storing only a provider-reported dollar total makes allocation and anomaly review impossible.
同时保存原始用量与计算所使用的 Price Version。用今天价格重算历史 Event 会破坏审计能力，而只保存供应商返回的美元总额又无法进行分摊与异常复核。
from decimal import Decimal
def request_cost(usage, price):
million = Decimal("1000000")
uncached = Decimal(usage["input_tokens"] - usage.get("cached_input_tokens", 0))
cached = Decimal(usage.get("cached_input_tokens", 0))
output = Decimal(usage["output_tokens"])
return {
"input_usd": uncached / million * Decimal(price["input_per_mtok"]),
"cached_input_usd": cached / million * Decimal(price.get("cached_input_per_mtok", 0)),
"output_usd": output / million * Decimal(price["output_per_mtok"]),
"price_version": price["version"],
"currency": "USD",
}
# Persist the original usage event and each calculated component.- Map native model IDs to prices with effective-from and effective-to timestamps.
- Keep provider fees, gateway fees, tools, storage, taxes, credits, and discounts as separate components.
- Reconcile daily ledger totals to provider exports before using the data for chargeback.
- 用 Effective-From 与 Effective-To 时间把原生模型 ID 映射到价格。
- 把供应商费、网关费、工具、存储、税费、Credits 与折扣作为独立组件。
- 用于内部计费前，每日将账本汇总与供应商导出对账。
Verified implementation reference: OpenTelemetry GenAI conventions.
实施参考已根据官方资料核验：OpenTelemetry GenAI conventions。
FAQ
Use them for estimates; reconcile billing with provider-native usage and invoices.
Attribute every attempt to the parent workload and mark its final disposition.
Only when justified and governed; usage tracking usually needs metadata, not content.
可用于估算；计费应与供应商原生用量和账单对账。
把每次尝试归属到父工作负载，并标记最终处置。
仅在有正当理由且受治理时；用量追踪通常只需元数据。