Metraipolis › archive › dc_ea2449a0bd50
RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature — arXiv:2608.27394
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.27394
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T15:20:45.711Z
- SHA-256 of the archived text
99e4b8d71758d4ffdda9aa9112b26220f3b484f75fb8c512dad4f7baf28feb00- Stable URL of this page
https://metraipolis.com/documento/dc_ea2449a0bd50
How to cite
RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature — arXiv:2608.27394 — arXiv (en), https://arxiv.org/abs/2608.27394 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea2449a0bd50 (Metraipolis archive dc_ea2449a0bd50, sha256 99e4b8d71758d4ffdda9aa9112b26220f3b484f75fb8c512dad4f7baf28feb00)
BibTeX
@misc{dcea2449a0bd50,
title = {RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature — arXiv:2608.27394},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.27394},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea2449a0bd50; sha256 99e4b8d71758d4ffdda9aa9112b26220f3b484f75fb8c512dad4f7baf28feb00},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27394. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction---zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision---previously used only for classification---to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers and LLM reranking helps further, but much room for improvement remains. Researchers blind to condition found the fine-tuned retriever's candidates more useful as inspirations. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues in scientific inspiration retrieval. We release full data, code, models, and annotations. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2608.27394 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T15:15:18.099Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).