Metraipolis › archive › dc_3ba3a0ea57fb
A Feasibility Condition for Confidence-Gated Tiered LLM Inference — arXiv:2611.05394
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.05394
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:16:33.928Z
- SHA-256 of the archived text
adc56ee60d7d7cc848ebc5a6b6eeb5076d1c7ed7d40de0303a24d6428f1a8837- Stable URL of this page
https://metraipolis.com/documento/dc_3ba3a0ea57fb
How to cite
A Feasibility Condition for Confidence-Gated Tiered LLM Inference — arXiv:2611.05394 — arXiv (en), https://arxiv.org/abs/2611.05394 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3ba3a0ea57fb (Metraipolis archive dc_3ba3a0ea57fb, sha256 adc56ee60d7d7cc848ebc5a6b6eeb5076d1c7ed7d40de0303a24d6428f1a8837)
BibTeX
@misc{dc3ba3a0ea57fb,
title = {A Feasibility Condition for Confidence-Gated Tiered LLM Inference — arXiv:2611.05394},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.05394},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3ba3a0ea57fb; sha256 adc56ee60d7d7cc848ebc5a6b6eeb5076d1c7ed7d40de0303a24d6428f1a8837},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05394. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tiered inference distributes large language model computation across a hierarchy of devices, from on-device and edge hardware to the datacenter, and is increasingly offered as the route to cheaper, lower-latency, and more private serving. Most such systems orchestrate frozen model checkpoints, and most decide what to resolve locally and what to escalate using a confidence signal produced by a forward-deployed model. We give a closed-form condition for when no threshold on that signal makes a tiered cascade worthwhile relative to sending every query to the backstop. Modeling the routing decision as a binary test and separating local from escalation cost through an architecture cost ratio $\kappa$, we derive a feasibility condition bracketed by a signal-free floor, below which a cascade is feasible at any signal quality, and a perfect-signal ceiling, above which none is. Read along the $\kappa$ axis the condition inverts: where local compute is billed, cost binds, matching a recent empirical finding; where it is sunk, on decentralized hardware, feasibility becomes unconditional and signal quality instead sets what the cascade achieves. We turn the condition into three design requirements that existing trained-in primitives satisfy only in isolation, and we propose, as future work, a co-designed architecture and a protocol to test it. The condition and the inversion hold independently of that architecture; they are the paper's contribution. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2611.05394 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:12:48.986Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).