Metraipolis › archive › dc_a2ba004ffb58

On the Limits of Test-Time Compute for Self-Evaluation — arXiv:2611.00350

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00350
License (as archived)
CC0-1.0
Archived
2026-11-03T10:45:51.390Z
SHA-256 of the archived text
7d23277b8ce47a7a9d19aac0cf803136f76875d2ee443247d68cee8320e583f3
Stable URL of this page
https://metraipolis.com/documento/dc_a2ba004ffb58

How to cite

On the Limits of Test-Time Compute for Self-Evaluation — arXiv:2611.00350 — arXiv (en), https://arxiv.org/abs/2611.00350 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a2ba004ffb58 (Metraipolis archive dc_a2ba004ffb58, sha256 7d23277b8ce47a7a9d19aac0cf803136f76875d2ee443247d68cee8320e583f3)
BibTeX
@misc{dca2ba004ffb58,
  title = {On the Limits of Test-Time Compute for Self-Evaluation — arXiv:2611.00350},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00350},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a2ba004ffb58; sha256 7d23277b8ce47a7a9d19aac0cf803136f76875d2ee443247d68cee8320e583f3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00350. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Self-verification and aggregation have emerged as popular approaches for bridging the gap between pass@1 and pass@N. However, increasing the number of candidates does not consistently narrow the gap between final-answer accuracy and pass@N. To understand this limitation, we use ranking error to measure how reliably models distinguish their correct generations from incorrect ones. We show that ranking error admits a bias-variance decomposition, with both components shaped by algorithmic design choices and inference compute. Across our experiments, additional compute reduces variance, while bias initially decreases before approaching a plateau. Guided by this analysis, we introduce Scorpio, a simple and general method that jointly scores randomly grouped candidates. Across math, code, and puzzle benchmarks, Scorpio improves verification, tree search, and aggregation. For verification, Scorpio improves accuracy over the strongest baseline by up to 7 percentage points while matching or exceeding that baseline's peak accuracy with 17-42% lower inference cost. For tree search, using Scorpio to score partial solutions and select the final answer improves accuracy by up to 10 percentage points over prior methods. Finally, Scorpio improves the scaling of aggregation-based methods by 11 percentage points on coding tasks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.00350 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:39:50.395Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).