Metraipolis › archive › dc_40f5ffa3cff1
Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence — arXiv:2609.05385
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.05385
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T22:23:48.125Z
- SHA-256 of the archived text
06b55868650703f77cb07b0a844d1bf770634dbe4e80430158470e92789aca9e- Stable URL of this page
https://metraipolis.com/documento/dc_40f5ffa3cff1
How to cite
Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence — arXiv:2609.05385 — arXiv (en), https://arxiv.org/abs/2609.05385 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40f5ffa3cff1 (Metraipolis archive dc_40f5ffa3cff1, sha256 06b55868650703f77cb07b0a844d1bf770634dbe4e80430158470e92789aca9e)
BibTeX
@misc{dc40f5ffa3cff1,
title = {Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence — arXiv:2609.05385},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.05385},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40f5ffa3cff1; sha256 06b55868650703f77cb07b0a844d1bf770634dbe4e80430158470e92789aca9e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05385. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
LLM decision components that can operate within agent workflows often produce action-relevant recommendations or judgements together with explanations. Operators may use the named factors to monitor a system, diagnose errors, or decide when to escalate an output. Such use assumes that the explanations agree with the component's observable decision behaviour. We test two interpretations of the named factors: necessity, meaning that changing a factor would change the output, and sufficiency, meaning that retaining it while removing other changeable information would preserve the output. We evaluate these interpretations in two synthetic use cases: recommending advisors to clients and judging prompts for harmfulness or risk. Models return an output and the top three factors that most influenced it. Controlled black-box interventions estimate a necessity score for each factor by measuring how often changing it changes the output, and a sufficiency score by measuring how often retaining it preserves the output. Across eight models from the Claude, GPT, and Gemini families, the mean Spearman correlations between the cited ranking and the necessity and sufficiency scores are 0.349 and 0.354 for advisor recommendation, and 0.431 and 0.580 for prompt monitoring. Furthermore, an uncited factor scores above the lowest-scoring cited factor in 57.6% of advisor responses under necessity and 58.1% under sufficiency; the corresponding prompt-monitoring rates are 25.8% and 8.9%. The cited top three contain useful information but do not reliably identify the three factors with the strongest measured influence under necessity or sufficiency. The framework provides a black-box reliability check for explanations used in agent oversight while remaining scoped to individual LLM decisions. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.05385 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T22:18:11.466Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).