Metraipolis › archive › dc_5ea85b910b2b

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict — arXiv:2610.12360

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.12360
License (as archived): CC0-1.0
Archived: 2026-10-09T05:50:43.219Z
SHA-256 of the archived text: dd307cc9135ecb10d22708c4bb80f9306daf8927daac3b3093674a1a15d7fdfb
Stable URL of this page: https://metraipolis.com/documento/dc_5ea85b910b2b

How to cite

Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict — arXiv:2610.12360 — arXiv (en), https://arxiv.org/abs/2610.12360 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5ea85b910b2b (Metraipolis archive dc_5ea85b910b2b, sha256 dd307cc9135ecb10d22708c4bb80f9306daf8927daac3b3093674a1a15d7fdfb)

BibTeX

@misc{dc5ea85b910b2b,
  title = {Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict — arXiv:2610.12360},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.12360},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5ea85b910b2b; sha256 dd307cc9135ecb10d22708c4bb80f9306daf8927daac3b3093674a1a15d7fdfb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12360. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution. We operationalize EH through three trajectory-level behavioral dimensions: Identify, Solve, and Escalate (ISE). Through knowledge conflict, situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we evaluate two conflict settings: (1) controlled conflict and (2) naturally occurring conflict during multi-step agentic execution, each paired with matched no-conflict controls. Evaluating four agents, we find that higher task accuracy does not necessarily correspond to greater epistemic humility: some high-accuracy configurations recognize conflicts during execution but do not communicate unresolved uncertainty in their incorrect final answers. Trajectory-level analysis further reveals that agents frequently detect conflicts in early steps of execution but fail to maintain or resolve them in later steps. Finally, we show that model-level interventions can improve EH, but often at the cost of task accuracy, suggesting that epistemic humility emerges from the interaction among the backbone model, agent harness, and evaluation environment.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.12360 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:01.110Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).