Metraipolis › archive › dc_8a26f73dc7e8

Inspire: Benchmarking Scientific Literature Search for Open Research Problems — arXiv:2609.33233

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33233
License (as archived)
CC0-1.0
Archived
2026-10-07T21:01:11.225Z
SHA-256 of the archived text
74b162334d25e3eb6cdfaa4218677f252ebf605c28dd903e804a3e7e710a16ef
Stable URL of this page
https://metraipolis.com/documento/dc_8a26f73dc7e8

How to cite

Inspire: Benchmarking Scientific Literature Search for Open Research Problems — arXiv:2609.33233 — arXiv (en), https://arxiv.org/abs/2609.33233 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8a26f73dc7e8 (Metraipolis archive dc_8a26f73dc7e8, sha256 74b162334d25e3eb6cdfaa4218677f252ebf605c28dd903e804a3e7e710a16ef)
BibTeX
@misc{dc8a26f73dc7e8,
  title = {Inspire: Benchmarking Scientific Literature Search for Open Research Problems — arXiv:2609.33233},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33233},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8a26f73dc7e8; sha256 74b162334d25e3eb6cdfaa4218677f252ebf605c28dd903e804a3e7e710a16ef},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33233. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Scientific literature search often begins with an open research problem rather than a known target paper or a fixed candidate set. We introduce INSPIRE, a benchmark for evaluating agents that search prior literature to make progress on solution-redacted research problems. Each instance pairs a research brief with a target-specific cutoff three months before a later paper and evaluates ranked outputs against graded cited antecedents from that paper's realized research lineage. Search proceeds over an open corpus, while the identity of the target paper and membership of its cited antecedents remain hidden from the agent. Beyond end-to-end retrieval quality, INSPIRE uses logged search trajectories to distinguish three coupled stages: resource exposure, whether useful antecedents are surfaced during search; selection, whether exposed antecedents are retained; and ranking, how effectively retained papers are ordered. Across 476 computer-science targets under a shared search interface and budget, the strongest evaluated agent achieves 0.284 nDCG@10. Results show that current agents more readily recover an isolated antecedent than assemble a broader portfolio of relevant prior work. The stagewise analysis identifies resource exposure as the largest observed bottleneck, with further losses in selection and ranking. We additionally construct replay-valid hindsight demonstrations and show that they improve held-out search without changing test-time information, establishing that the benchmark provides an actionable learning signal. INSPIRE therefore enables both end-to-end comparison and stage-resolved diagnosis in a setting where the agent must construct its own working criterion of relevance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Information Retrieval (cs.IR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.33233 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.588Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).