Metraipolis › archive › dc_a4abe3d6bd67

Inference for Reasoning Models: Practical Lessons from Evaluation and Serving — arXiv:2611.10845

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10845
License (as archived)
CC0-1.0
Archived
2026-11-10T18:41:15.649Z
SHA-256 of the archived text
f29ac38b8a37b86c6e12e7edb1ca879bee63fbb46ecd8dae5e0e74fd16eaa5f8
Stable URL of this page
https://metraipolis.com/documento/dc_a4abe3d6bd67

How to cite

Inference for Reasoning Models: Practical Lessons from Evaluation and Serving — arXiv:2611.10845 — arXiv (en), https://arxiv.org/abs/2611.10845 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a4abe3d6bd67 (Metraipolis archive dc_a4abe3d6bd67, sha256 f29ac38b8a37b86c6e12e7edb1ca879bee63fbb46ecd8dae5e0e74fd16eaa5f8)
BibTeX
@misc{dca4abe3d6bd67,
  title = {Inference for Reasoning Models: Practical Lessons from Evaluation and Serving — arXiv:2611.10845},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10845},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a4abe3d6bd67; sha256 f29ac38b8a37b86c6e12e7edb1ca879bee63fbb46ecd8dae5e0e74fd16eaa5f8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10845. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Running a language model involves more than its trained weights. The same model can receive very different benchmark scores, produce different answers, or cost more to serve depending on how it is evaluated and run. In one experiment in this book, changing only the maximum response length moved accuracy on a math benchmark from 0.08% to 93.18%. This book explains how inference settings shape the results we report and the systems we deploy. We begin with evaluation, including generation limits, answer extraction, scoring methods, and quantization. We then examine why outputs can change across batches, attention backends, and caching, even when sampling is disabled. The next chapters cover serving performance, including batching, memory capacity, latency, throughput, and cost. We also study structured outputs, tool calls, reasoning budgets, and long-context tasks, showing what these controls and benchmarks do and do not establish. The book concludes with practical guidance for recording inference runs, checking evaluations, estimating capacity, and comparing systems fairly. Written for researchers and engineers working with language models, it aims to make inference results easier to understand, reproduce, and trust.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10845 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.506Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).