Metraipolis › archive › dc_46ee07dda53a

Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents — arXiv:2608.29685

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29685
License (as archived)
CC0-1.0
Archived
2026-10-17T08:36:53.399Z
SHA-256 of the archived text
42012211ee1b7206e6c2e110061d0e798fe3ae681fa1b8558c8f7f68302640d8
Stable URL of this page
https://metraipolis.com/documento/dc_46ee07dda53a

How to cite

Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents — arXiv:2608.29685 — arXiv (en), https://arxiv.org/abs/2608.29685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46ee07dda53a (Metraipolis archive dc_46ee07dda53a, sha256 42012211ee1b7206e6c2e110061d0e798fe3ae681fa1b8558c8f7f68302640d8)
BibTeX
@misc{dc46ee07dda53a,
  title = {Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents — arXiv:2608.29685},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29685},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46ee07dda53a; sha256 42012211ee1b7206e6c2e110061d0e798fe3ae681fa1b8558c8f7f68302640d8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Early failure prediction is important for long-horizon agents, as it enables timely intervention and can reduce inference and tool-use costs. Uncertainty quantification, such as verbal confidence and perplexity, offers a promising approach to detecting agent failures; however, it has not been explored whether these signals retain their discriminative power during the intermediate stages of long-horizon execution. We evaluate mainstream uncertainty signals on deep-research tasks and find that verbal confidence reliably distinguishes failures at trajectory completion, achieving a mean AUROC of 0.85, whereas all evaluated signals offer limited predictive value earlier in execution, with none exceeding a mean AUROC of 0.60 at 50% trajectory progress. We identify an underlying mechanism explaining this gap: path switching, where agents frequently abandon their current search direction in-trajectory, breaking the link between early signal and final outcome. These findings challenge the assumption that intermediate uncertainty can reliably guide early intervention. They also motivate a practical recommendation for agent harnesses in deep-research settings: use final-step confidence to decide whether to restart, an approach that our experiments find more effective than in-trajectory intervention.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.29685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T08:13:26.762Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).