Track LLM Usage & Cost
Track LLM usage and cost by joining provider bills with request-level model, token, retry, route, and price evidence, then reconciling both.
Published: March 24, 2026 · By the QVeris Team
TL;DR
Link app trace, gateway request, provider attempt, response and tool-call IDs.
Record success, retry and fallback usage rather than only the final response.
Map units for analysis while retaining native fields and billing dimensions.
Compare internal ledgers with provider exports and explain every material difference.
The evidence model
Track usage at provider-attempt level, then roll it up to the user request and business workload. For every attempt, store tenant, project, environment, workload, route, provider, immutable model version, native request ID, input and output units, cache dimensions, retries, latency, status and allocation tags. Keep both provider-native values and normalized fields so later reporting does not erase how the source actually measured usage.
A usage event is not yet a cost. Join it to a versioned price record with currency, unit, effective time and pricing source. This matters when cached input, reasoning units, long-context tiers or priority processing have different rates. Never recalculate old events using today's price without preserving the historical rate used for the original estimate.
Treat tool calls, media processing, grounding, storage, batch and priority modes as separate billable dimensions when the provider exposes them. Do not compress them into a guessed token total. When a response omits usage—common for some failed or interrupted requests—mark it as unknown and reconcile from provider exports rather than silently recording zero.
Usage data sources
| Source | Best fit | Verify before choosing |
|---|---|---|
| Response usage | Low-latency product analytics and per-request evidence. | Missing responses, streams, retries and provider-specific fields. |
| Gateway ledger | Cross-provider normalization and tenant allocation. | Schema versioning, duplicate events and retry ownership. |
| Provider usage export | Authoritative provider-side aggregation. | Time zones, delayed data, model aliases and billing cutoffs. |
| Invoice or contract | Financial settlement and audit. | Credits, taxes, commitments, special tiers and non-token charges. |
Ledger design checklist
Append time-ordered usage evidence; correct with adjustment events.
Require tenant, environment, team, product and workload ownership.
Store native unit name, value, source and normalization version.
Avoid logging prompts or outputs when usage metadata is sufficient.
Build a reconciliation loop
Operational telemetry gives fast estimates; provider exports and invoices provide later financial evidence. Reconciliation connects the two. Define tolerances by provider and dimension, because rounding, delayed usage, currency conversion and invoice adjustments can create legitimate differences.
- Sample traces and verify every request, hop, attempt and billable action has a stable identifier.
- Replay retries, fallbacks, cache hits, long streams, cancellation and failed responses.
- Compare daily ledger totals with provider usage exports by model, project, region and billing dimension.
- Investigate missing usage, duplicate ingestion, late events and price-version mismatches before closing a period.
- Reconcile invoices and document matched, explained and unexplained deltas with owner and resolution status.
Join operational traces with a finance ledger
The application emits a parent trace ID and business allocation tags. The gateway creates a request ID and one child attempt for every provider call, including retries and fallbacks. Usage events enter an append-only ledger with a deterministic event key, source timestamp and ingestion timestamp so delayed or repeated delivery can be handled without double counting.
A pricing service resolves the rate effective at the event time and produces an estimate linked to the original usage record. A reconciliation job later aligns provider exports and invoices, then appends adjustments and supporting evidence instead of rewriting history. Dashboards can show request cost quickly while finance retains an auditable path from invoice line to workload.
Production rule: never report complete model cost without counting retries, fallbacks, unknown usage and non-token billable dimensions.
Trace tool and live-data charges separately
QVeris calls external APIs, tools, services and live data whose commercial units may be requests, records, seats, data packages or provider-specific credits rather than tokens. Propagate the parent trace, tenant and workload tags, but record capability ID, external provider evidence, unit, quantity and price version in a separate tool ledger.
Join model and tool ledgers only for workflow reporting. Keeping the source records separate prevents an expensive data lookup from being mislabeled as model spend and makes retries visible: one model retry may reuse an existing tool result, while another workflow may legitimately refresh it because the data freshness window expired.
Calculate cost from a versioned price catalog
Store raw usage and the price version used for calculation. Recomputing historical events with today's prices destroys auditability, while storing only a provider-reported dollar total makes allocation and anomaly review impossible.
from decimal import Decimal
def request_cost(usage, price):
million = Decimal("1000000")
uncached = Decimal(usage["input_tokens"] - usage.get("cached_input_tokens", 0))
cached = Decimal(usage.get("cached_input_tokens", 0))
output = Decimal(usage["output_tokens"])
return {
"input_usd": uncached / million * Decimal(price["input_per_mtok"]),
"cached_input_usd": cached / million * Decimal(price.get("cached_input_per_mtok", 0)),
"output_usd": output / million * Decimal(price["output_per_mtok"]),
"price_version": price["version"],
"currency": "USD",
}
# Persist the original usage event and each calculated component.- Map native model IDs to prices with effective-from and effective-to timestamps.
- Keep provider fees, gateway fees, tools, storage, taxes, credits, and discounts as separate components.
- Reconcile daily ledger totals to provider exports before using the data for chargeback.
Verified implementation reference: OpenTelemetry GenAI conventions.
Continue with related implementation guides
Use these adjacent guides to compare the next architectural decision, implementation boundary, or operating control without starting the research again.
FAQ
Use them for estimates; reconcile billing with provider-native usage and invoices.
Attribute every attempt to the parent workload and mark its final disposition.
Only when justified and governed; usage tracking usually needs metadata, not content.
Official sources and further reading
TL;DR
Link app trace, gateway request, provider attempt, response and tool-call IDs.
Record success, retry and fallback usage rather than only the final response.
Map units for analysis while retaining native fields and billing dimensions.
Compare internal ledgers with provider exports and explain every material difference.
The evidence model
Track usage at provider-attempt level, then roll it up to the user request and business workload. For every attempt, store tenant, project, environment, workload, route, provider, immutable model version, native request ID, input and output units, cache dimensions, retries, latency, status and allocation tags. Keep both provider-native values and normalized fields so later reporting does not erase how the source actually measured usage.
A usage event is not yet a cost. Join it to a versioned price record with currency, unit, effective time and pricing source. This matters when cached input, reasoning units, long-context tiers or priority processing have different rates. Never recalculate old events using today's price without preserving the historical rate used for the original estimate.
Treat tool calls, media processing, grounding, storage, batch and priority modes as separate billable dimensions when the provider exposes them. Do not compress them into a guessed token total. When a response omits usage—common for some failed or interrupted requests—mark it as unknown and reconcile from provider exports rather than silently recording zero.
Usage data sources
| Source | Best fit | Verify before choosing |
|---|---|---|
| Response usage | Low-latency product analytics and per-request evidence. | Missing responses, streams, retries and provider-specific fields. |
| Gateway ledger | Cross-provider normalization and tenant allocation. | Schema versioning, duplicate events and retry ownership. |
| Provider usage export | Authoritative provider-side aggregation. | Time zones, delayed data, model aliases and billing cutoffs. |
| Invoice or contract | Financial settlement and audit. | Credits, taxes, commitments, special tiers and non-token charges. |
Ledger design checklist
Append time-ordered usage evidence; correct with adjustment events.
Require tenant, environment, team, product and workload ownership.
Store native unit name, value, source and normalization version.
Avoid logging prompts or outputs when usage metadata is sufficient.
Build a reconciliation loop
Operational telemetry gives fast estimates; provider exports and invoices provide later financial evidence. Reconciliation connects the two. Define tolerances by provider and dimension, because rounding, delayed usage, currency conversion and invoice adjustments can create legitimate differences.
- Sample traces and verify every request, hop, attempt and billable action has a stable identifier.
- Replay retries, fallbacks, cache hits, long streams, cancellation and failed responses.
- Compare daily ledger totals with provider usage exports by model, project, region and billing dimension.
- Investigate missing usage, duplicate ingestion, late events and price-version mismatches before closing a period.
- Reconcile invoices and document matched, explained and unexplained deltas with owner and resolution status.
Join operational traces with a finance ledger
The application emits a parent trace ID and business allocation tags. The gateway creates a request ID and one child attempt for every provider call, including retries and fallbacks. Usage events enter an append-only ledger with a deterministic event key, source timestamp and ingestion timestamp so delayed or repeated delivery can be handled without double counting.
A pricing service resolves the rate effective at the event time and produces an estimate linked to the original usage record. A reconciliation job later aligns provider exports and invoices, then appends adjustments and supporting evidence instead of rewriting history. Dashboards can show request cost quickly while finance retains an auditable path from invoice line to workload.
Production rule: never report complete model cost without counting retries, fallbacks, unknown usage and non-token billable dimensions.
Trace tool and live-data charges separately
QVeris calls external APIs, tools, services and live data whose commercial units may be requests, records, seats, data packages or provider-specific credits rather than tokens. Propagate the parent trace, tenant and workload tags, but record capability ID, external provider evidence, unit, quantity and price version in a separate tool ledger.
Join model and tool ledgers only for workflow reporting. Keeping the source records separate prevents an expensive data lookup from being mislabeled as model spend and makes retries visible: one model retry may reuse an existing tool result, while another workflow may legitimately refresh it because the data freshness window expired.
Calculate cost from a versioned price catalog
Store raw usage and the price version used for calculation. Recomputing historical events with today's prices destroys auditability, while storing only a provider-reported dollar total makes allocation and anomaly review impossible.
- Map native model IDs to prices with effective-from and effective-to timestamps.
- Keep provider fees, gateway fees, tools, storage, taxes, credits, and discounts as separate components.
- Reconcile daily ledger totals to provider exports before using the data for chargeback.
Verified implementation reference: OpenTelemetry GenAI conventions.
Continue with related implementation guides
Use these adjacent guides to compare the next architectural decision, implementation boundary, or operating control without starting the research again.
Use them for estimates; reconcile billing with provider-native usage and invoices.
Attribute every attempt to the parent workload and mark its final disposition.
Only when justified and governed; usage tracking usually needs metadata, not content.