Metraipolis › archive › dc_f14ac61df015
Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01718
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:52:42.308Z
- SHA-256 of the archived text
c6f6a6ecbe0dafbb42f12b47cddc6aaa9441592017d71ca97758c47bfc58441e- Stable URL of this page
https://metraipolis.com/documento/dc_f14ac61df015
How to cite
Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718 — arXiv (en), https://arxiv.org/abs/2611.01718 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f14ac61df015 (Metraipolis archive dc_f14ac61df015, sha256 c6f6a6ecbe0dafbb42f12b47cddc6aaa9441592017d71ca97758c47bfc58441e)
BibTeX
@misc{dcf14ac61df015,
title = {Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01718},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f14ac61df015; sha256 c6f6a6ecbe0dafbb42f12b47cddc6aaa9441592017d71ca97758c47bfc58441e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01718. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Task success, retrieval quality, and cost do not show whether an LLM agent used a delivered memory or what followed from that use. We introduce MemUse, which evaluates memory at its point of use in recorded trajectories: each memory delivery is compared with what the agent could already see and followed through the agent's later actions and their consequences. An agentic judge rates each delivered entry's relevance, accuracy, novelty, influence, and observed effect; programmatic checks enforce complete reading of cited evidence, and fixed rules aggregate the ratings into verdicts linked to memory histories and delivery cost. On 8,000 SWE-bench Verified episodes from three memory systems, resolve rates stay within 0.4 percentage points of a no-memory baseline, yet MemUse shows that most delivered entries were irrelevant, most relevant entries went unused, and in two systems most deliveries restated content the agent could still see. It also surfaces effects that outcomes hide: 23.2\% of episodes with beneficial memory use still failed, and 5 of 16 with harmful use still succeeded. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.01718 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:49:38.314Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).