Metraipolis › archive › dc_fd2b54dbfb99

Confident Fools or Careful Searchers? A Behavioural Study of When LLMs Search Beyond Their Static Internal Memory — arXiv:2610.41124

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41124
License (as archived): CC0-1.0
Archived: 2026-11-02T07:08:40.013Z
SHA-256 of the archived text: 99a28e4c0f2f5a9b6535957d38ba21cb92a31c3f15f8d19aa4f35db4e8f3fd1d
Stable URL of this page: https://metraipolis.com/documento/dc_fd2b54dbfb99

How to cite

Confident Fools or Careful Searchers? A Behavioural Study of When LLMs Search Beyond Their Static Internal Memory — arXiv:2610.41124 — arXiv (en), https://arxiv.org/abs/2610.41124 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fd2b54dbfb99 (Metraipolis archive dc_fd2b54dbfb99, sha256 99a28e4c0f2f5a9b6535957d38ba21cb92a31c3f15f8d19aa4f35db4e8f3fd1d)

BibTeX

@misc{dcfd2b54dbfb99,
  title = {Confident Fools or Careful Searchers? A Behavioural Study of When LLMs Search Beyond Their Static Internal Memory — arXiv:2610.41124},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41124},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fd2b54dbfb99; sha256 99a28e4c0f2f5a9b6535957d38ba21cb92a31c3f15f8d19aa4f35db4e8f3fd1d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41124. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agentic LLM assistants must repeatedly decide whether to answer from memory or to call a tool such as web search. Unnecessary searches cost latency and money; missing ones yield confident but unsupported answers. Prior work mostly measures verbalised confidence or trains new adaptive-retrieval methods, and rarely audits the search decisions that deployed products actually make. We introduce a benchmark of 400 multiple-choice questions that combines widely known inside-knowledge (IK) questions, which should be answered directly, with outside-knowledge (OK) questions about facts from news published after the models' training cut-offs (to our knowledge), which should trigger search or, without tools, abstention. Each item offers an abstain option, and two action-level scores, NAS and KIS, reward searching on OK questions and restraint on IK questions. Across five commercial LLMs with and without their vendors' web search, GPT-5 mini and Grok 4 Fast search on over 90% of OK but only 1% of IK questions, whereas Mistral Small searches on just 20% of OK questions, often abstaining instead of using its tool; one product's search indicator proved unusable. Searching is partly triggered by explicit temporal cues: on uncued OK questions, the NAS of the two selective models falls from about 0.72 to 0.57--0.58. The scores thus measure the search triggering of specific products, not metacognition in a strict sense.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41124 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:06:59.304Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).