Metraipolis › archive › dc_7223371cb35f

BlindToChange: AI Agents Can Find Changes When Asked But Do They Look When Needed? — arXiv:2610.41435

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41435
License (as archived)
CC0-1.0
Archived
2026-11-02T07:01:09.895Z
SHA-256 of the archived text
1898f804dd787d4972b8b86079fd8f2c1016906f219ff20ee491475280a357f1
Stable URL of this page
https://metraipolis.com/documento/dc_7223371cb35f

How to cite

BlindToChange: AI Agents Can Find Changes When Asked But Do They Look When Needed? — arXiv:2610.41435 — arXiv (en), https://arxiv.org/abs/2610.41435 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7223371cb35f (Metraipolis archive dc_7223371cb35f, sha256 1898f804dd787d4972b8b86079fd8f2c1016906f219ff20ee491475280a357f1)
BibTeX
@misc{dc7223371cb35f,
  title = {BlindToChange: AI Agents Can Find Changes When Asked But Do They Look When Needed? — arXiv:2610.41435},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41435},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7223371cb35f; sha256 1898f804dd787d4972b8b86079fd8f2c1016906f219ff20ee491475280a357f1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41435. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Given that AI agents can search the web broadly and deeply in real time, it is tempting to conclude that their outputs account for recent world events. We show that this conclusion does not hold in practice. While agents can search and easily find recent changes, the correctness of their responses depends on whether they search and what they search for. Popular agentic search benchmarks like BrowseComp cannot expose failures wherein agents do not recognize their own assumptions that should be verified, since these benchmarks explicitly state what needs to be found. In contrast, many user requests state only the final information need (e.g., an itinerary for a trip), leaving the agent to discover relevant changes (e.g., the temporary closure of a museum) and account for their consequences. To highlight such gaps, we introduce BlindToChange, a benchmark that measures whether agents proactively discover recent events that affect their responses. Our benchmark comprises requests for recommendations where an obvious candidate is no longer applicable due to a recent real-world event. These events are drawn automatically from Wikidata, allowing BlindToChange to be periodically refreshed. Strikingly, we find a big gap -- as high as 27.5 points, even at high reasoning effort -- in systems' performance, when the recent event is not explicitly mentioned compared to when we ask about it directly. Analyzing search trajectories, we learn that more than half of the failures can be attributed to the fact that agents do not directly search to verify whether a relevant fact has changed. Our benchmark highlights, and tracks, a new axis of progress, wherein systems account for recent information that is not explicitly mentioned in the user request.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41435 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T06:56:58.394Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).