Metraipolis › archive › dc_22fa1b867906
Dr. Free: You Don't Need Difficulty Rewards for Self-Evolving Search Agents — arXiv:2609.33565
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33565
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:56:55.799Z
- SHA-256 of the archived text
7bd9fe82a82df51aa23eb4a920a764109ef02ca8516a9156cd69a5975e34d475- Stable URL of this page
https://metraipolis.com/documento/dc_22fa1b867906
How to cite
Dr. Free: You Don't Need Difficulty Rewards for Self-Evolving Search Agents — arXiv:2609.33565 — arXiv (en), https://arxiv.org/abs/2609.33565 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_22fa1b867906 (Metraipolis archive dc_22fa1b867906, sha256 7bd9fe82a82df51aa23eb4a920a764109ef02ca8516a9156cd69a5975e34d475)
BibTeX
@misc{dc22fa1b867906,
title = {Dr. Free: You Don't Need Difficulty Rewards for Self-Evolving Search Agents — arXiv:2609.33565},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33565},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_22fa1b867906; sha256 7bd9fe82a82df51aa23eb4a920a764109ef02ca8516a9156cd69a5975e34d475},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33565. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A central limitation of current data-free self-evolution methods for training search agents is their reliance on difficulty-based proposer rewards. These methods reward a proposer for generating questions that challenge a co-evolving solver, using solver difficulty as a proxy for question quality. Yet difficulty alone is insufficient to distinguish questions that require cross-passage evidence from those that are answerable via simpler shortcuts. In addition, measuring difficulty demands repeated solver rollouts for every candidate question, leading to substantial computational costs. In this paper, we introduce \methodname, the first self-evolving search framework that eliminates difficulty-based proposer rewards and directly optimizes for evidence necessity relative to shortcut contexts. Dr. Free samples relational chains from a knowledge graph and pairs them with aligned passages, giving question generation an explicit multi-hop structure. A generated question receives a positive information-gain reward only when the likelihood of the target answer under the complete evidence passages exceeds the maximum likelihood under all evaluated shortcut contexts. Because this signal is computed from teacher-forced likelihoods, it removes the need for pass-rate estimation and reduces proposer training time by over $7\times$. Experiments on seven open-domain QA benchmarks show that Dr. Free outperforms prior data-free search agents and the supervised baseline, with large improvements on multi-hop QA benchmarks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.33565 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:01.300Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).