Metraipolis › archive › dc_fbf31ee5e075
Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions — arXiv:2610.37624
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.37624
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-28T03:02:07.824Z
- SHA-256 of the archived text
d4cfc84b76719e2196f9f24427e4a5844e2b0797350052d417062f3a13e22c43- Stable URL of this page
https://metraipolis.com/documento/dc_fbf31ee5e075
How to cite
Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions — arXiv:2610.37624 — arXiv (en), https://arxiv.org/abs/2610.37624 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fbf31ee5e075 (Metraipolis archive dc_fbf31ee5e075, sha256 d4cfc84b76719e2196f9f24427e4a5844e2b0797350052d417062f3a13e22c43)
BibTeX
@misc{dcfbf31ee5e075,
title = {Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions — arXiv:2610.37624},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.37624},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fbf31ee5e075; sha256 d4cfc84b76719e2196f9f24427e4a5844e2b0797350052d417062f3a13e22c43},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.37624. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
When an LLM group's discussion is compressed into a short memory for a final decision maker, what should the memory keep? Two natural heuristics keep the rarest claims or the claims relevant to the decision criteria. On hidden-profile tasks both look attractive, because how widely a fact is held is confounded with whether it helps. We introduce CrossProfile, a diagnostic benchmark of 60 templates with automatically verified answer keys, in which the ownership of supporting, decoy, and irrelevant facts is crossed while their text is held fixed (480 tasks). Across Qwen2.5-7B, Llama-3.1-8B, and two DeepSeek models, both heuristics fail in predictable places. Selecting by relevance readmits decoys, because decoys are on topic: even a perfect relevance filter loses 23 to 31 points to rarest-first selection in the classic hidden-profile configuration. Rarest-first selection is no better than random selection on average and wins only where the rare facts happen to be the useful ones. In both cases accuracy follows the balance of evidence the memory keeps, not the policy that kept it. The decision model adds a third failure, a form of social proof: it gives extra weight to facts attributed to many agents. Removing the attribution tags, or reassigning them at random with line order fixed, raises accuracy in the classic configuration by 13 to 24 points. In natural discussions the evidence is lost before any memory acts: the complete transcript yields lower accuracy than no notes at all. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Multiagent Systems (cs.MA); Social and Information Networks (cs.SI). Pagina del lavoro: https://arxiv.org/abs/2610.37624 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-28T03:01:03.540Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).