Metraipolis › archive › dc_46ee07dda53a
Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents — arXiv:2608.29685
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.29685
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T08:36:53.399Z
- SHA-256 of the archived text
42012211ee1b7206e6c2e110061d0e798fe3ae681fa1b8558c8f7f68302640d8- Stable URL of this page
https://metraipolis.com/documento/dc_46ee07dda53a
How to cite
Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents — arXiv:2608.29685 — arXiv (en), https://arxiv.org/abs/2608.29685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46ee07dda53a (Metraipolis archive dc_46ee07dda53a, sha256 42012211ee1b7206e6c2e110061d0e798fe3ae681fa1b8558c8f7f68302640d8)
BibTeX
@misc{dc46ee07dda53a,
title = {Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents — arXiv:2608.29685},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.29685},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46ee07dda53a; sha256 42012211ee1b7206e6c2e110061d0e798fe3ae681fa1b8558c8f7f68302640d8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Early failure prediction is important for long-horizon agents, as it enables timely intervention and can reduce inference and tool-use costs. Uncertainty quantification, such as verbal confidence and perplexity, offers a promising approach to detecting agent failures; however, it has not been explored whether these signals retain their discriminative power during the intermediate stages of long-horizon execution. We evaluate mainstream uncertainty signals on deep-research tasks and find that verbal confidence reliably distinguishes failures at trajectory completion, achieving a mean AUROC of 0.85, whereas all evaluated signals offer limited predictive value earlier in execution, with none exceeding a mean AUROC of 0.60 at 50% trajectory progress. We identify an underlying mechanism explaining this gap: path switching, where agents frequently abandon their current search direction in-trajectory, breaking the link between early signal and final outcome. These findings challenge the assumption that intermediate uncertainty can reliably guide early intervention. They also motivate a practical recommendation for agent harnesses in deep-research settings: use final-step confidence to decide whether to restart, an approach that our experiments find more effective than in-trajectory intervention. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.29685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T08:13:26.762Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).