Metraipolis › archive › dc_cf78fc36bb75

Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services — arXiv:2611.01371

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.01371
License (as archived): CC0-1.0
Archived: 2026-11-03T18:26:38.291Z
SHA-256 of the archived text: cdcd64810f316515b355e74383ebd27f3bd7434f170d81be1e2f897d6b81a3a9
Stable URL of this page: https://metraipolis.com/documento/dc_cf78fc36bb75

How to cite

Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services — arXiv:2611.01371 — arXiv (en), https://arxiv.org/abs/2611.01371 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cf78fc36bb75 (Metraipolis archive dc_cf78fc36bb75, sha256 cdcd64810f316515b355e74383ebd27f3bd7434f170d81be1e2f897d6b81a3a9)

BibTeX

@misc{dccf78fc36bb75,
  title = {Token-as-a-Utility: A Model-Agnostic Architecture for Metering and Provisioning AI Inference Services — arXiv:2611.01371},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01371},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cf78fc36bb75; sha256 cdcd64810f316515b355e74383ebd27f3bd7434f170d81be1e2f897d6b81a3a9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01371. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Token-based subscriptions are emerging as a commercial model for large language model inference, but extending them into interoperable utility services requires comparable consumption accounting. Raw token counts do not consistently measure inference resource use because demand varies with model architecture, context length, tokenisation, and concurrent serving conditions. This paper presents Token-as-a-Utility, a model-agnostic architecture integrating inference metering, service-level-aware provisioning, and inter-operator settlement. We prove that, under batched serving, no per-request meter can be additive, independent of co-resident traffic, and exactly cost-reflective at once. This motivates separating provisioned capacity, metered consumption, and contracted quality into distinct service products. Consumption is expressed in Reference Inference Units (RIUs) derived from prefill and decode demand, while occupancy-indexed operator ratings recover shared serving costs without altering the common unit. Versioned inference records support auditing and re-rating across operators; software-enforced capacity leases and quality-constrained routing enable provisioning on shared accelerators. The resource model is measured on three NVIDIA RTX 5000 Ada GPUs, and multi-tenant behaviour is evaluated with a trace-driven model calibrated to them. For the reference model, occupancy-indexed rating limits statement energy error to 9% under measured traffic, compared with up to 70.5% under fixed-occupancy rating. In calibrated multi-tenant experiments, tenant mispricing decreases from 7.36 times under flat token counting to 2.39 times with RIUs and 1.06 times after occupancy-based reconciliation. These findings demonstrate how portable inference accounting can be combined with load-dependent cost recovery and contract-based provisioning to support heterogeneous inference services across operators.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Networking and Internet Architecture (cs.NI); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2611.01371 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:22.813Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).