Metraipolis › archive › dc_5b4097399998

LongSearchEval: Evaluating Long-Horizon Web Search Capabilities of Large Language Models via a Trajectory-Aware Agent Framework with Failure-Node Localization — arXiv:2610.35150

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.35150
License (as archived)
CC0-1.0
Archived
2026-10-27T07:58:28.098Z
SHA-256 of the archived text
4d8b5fab793c19cc4cc10343668f0474fc529f14ddacb9247d7134e72ec45345
Stable URL of this page
https://metraipolis.com/documento/dc_5b4097399998

How to cite

LongSearchEval: Evaluating Long-Horizon Web Search Capabilities of Large Language Models via a Trajectory-Aware Agent Framework with Failure-Node Localization — arXiv:2610.35150 — arXiv (en), https://arxiv.org/abs/2610.35150 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5b4097399998 (Metraipolis archive dc_5b4097399998, sha256 4d8b5fab793c19cc4cc10343668f0474fc529f14ddacb9247d7134e72ec45345)
BibTeX
@misc{dc5b4097399998,
  title = {LongSearchEval: Evaluating Long-Horizon Web Search Capabilities of Large Language Models via a Trajectory-Aware Agent Framework with Failure-Node Localization — arXiv:2610.35150},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.35150},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5b4097399998; sha256 4d8b5fab793c19cc4cc10343668f0474fc529f14ddacb9247d7134e72ec45345},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35150. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-horizon web search requires LLM-based agents to integrate information across web pages through extended interactions under task-specific constraints. Evaluating this capability is important not only for measuring final answer quality, but also for assessing whether an agent acquires, verifies, and uses evidence reliably over extended interactions. Existing evaluations, however, mainly rely on simulated trajectories or on post-hoc trajectories reconstructed from final answers, rather than on retained real interaction traces, which limits the fidelity of behavioral assessment and weakens support for fine-grained error localization. To address this gap, we introduce LONGSEARCHEVAL, a trajectory-aware agent evaluation framework that combines a unified agent execution protocol with source constrained outcome verification and retrospective failure-node localization. Stage 1 verifies answer groups against agent-cited webpages without human-authored gold answers; Stage 2 audits normalized rounds and links flagged failure nodes to incorrect answer groups. We evaluate 16 models from eight families on 100 tasks, yielding 1,600 retained runs and 6,545 automatically flagged failure nodes. Trajectory collection and evaluation required 1,037 and 318 hours of cumulative runtime, respectively. Human validation yields 94.50% agreement on supplied answer-group labels and 98.11% feature-correspondence precision for audited round-group links. Together, outcome scores and linked trajectory evidence support controlled model comparison and retrospective inspection of answer-related content.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.35150 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.737Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).