Metraipolis › archive › dc_528ca383cc78

Confident Fools or Careful Searchers? A Behavioural Study of When LLMs Search Beyond Their Static Internal Memory — arXiv:2610.41124

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41124
License (as archived)
CC0-1.0
Archived
2026-11-03T07:31:21.683Z
SHA-256 of the archived text
0ee48d4e3c963ffe1ad7245ca5db7748871104c0382c8943f4d057c187bf2968
Stable URL of this page
https://metraipolis.com/documento/dc_528ca383cc78

How to cite

Confident Fools or Careful Searchers? A Behavioural Study of When LLMs Search Beyond Their Static Internal Memory — arXiv:2610.41124 — arXiv (en), https://arxiv.org/abs/2610.41124 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_528ca383cc78 (Metraipolis archive dc_528ca383cc78, sha256 0ee48d4e3c963ffe1ad7245ca5db7748871104c0382c8943f4d057c187bf2968)
BibTeX
@misc{dc528ca383cc78,
  title = {Confident Fools or Careful Searchers? A Behavioural Study of When LLMs Search Beyond Their Static Internal Memory — arXiv:2610.41124},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41124},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_528ca383cc78; sha256 0ee48d4e3c963ffe1ad7245ca5db7748871104c0382c8943f4d057c187bf2968},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41124. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agentic LLM assistants must repeatedly decide whether to answer from memory or to call a tool such as web search. Unnecessary searches cost latency and money; missing ones yield confident but unsupported answers. Prior work mostly measures verbalised confidence or trains new adaptive-retrieval methods, and rarely audits the search decisions that deployed products actually make. We introduce a benchmark of 400 multiple-choice questions that combines widely known inside-knowledge (IK) questions, which should be answered directly, with outside-knowledge (OK) questions about facts from news published after the models' training cut-offs (to our knowledge), which should trigger search or, without tools, abstention. Each item offers an abstain option, and two action-level scores, NAS and KIS, reward searching on OK questions and restraint on IK questions. Across five commercial LLMs with and without their vendors' web search, GPT-5 mini and Grok 4 Fast search on over 90% of OK but only 1% of IK questions, whereas Mistral Small searches on just 20% of OK questions, often abstaining instead of using its tool; one product's search indicator proved unusable. Searching is partly triggered by explicit temporal cues: on uncued OK questions, the NAS of the two selective models falls from about 0.72 to 0.57--0.58. The scores thus measure the search triggering of specific products, not metacognition in a strict sense.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41124 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:29:33.600Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).