Metraipolis › archive › dc_ec3a3974ef4c
Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01718
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:42:25.502Z
- SHA-256 of the archived text
5037eab1e35d1fdb95b890eced58808eb374185b1aa6959e01490e0475024981- Stable URL of this page
https://metraipolis.com/documento/dc_ec3a3974ef4c
How to cite
Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718 — arXiv (en), https://arxiv.org/abs/2611.01718 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec3a3974ef4c (Metraipolis archive dc_ec3a3974ef4c, sha256 5037eab1e35d1fdb95b890eced58808eb374185b1aa6959e01490e0475024981)
BibTeX
@misc{dcec3a3974ef4c,
title = {Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01718},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec3a3974ef4c; sha256 5037eab1e35d1fdb95b890eced58808eb374185b1aa6959e01490e0475024981},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01718. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Task success, retrieval quality, and cost do not show whether an LLM agent used a delivered memory or what followed from that use. We introduce MemUse, which evaluates memory at its point of use in recorded trajectories: each memory delivery is compared with what the agent could already see and followed through the agent's later actions and their consequences. An agentic judge rates each delivered entry's relevance, accuracy, novelty, influence, and observed effect; programmatic checks enforce complete reading of cited evidence, and fixed rules aggregate the ratings into verdicts linked to memory histories and delivery cost. On 8,000 SWE-bench Verified episodes from three memory systems, resolve rates stay within 0.4 percentage points of a no-memory baseline, yet MemUse shows that most delivered entries were irrelevant, most relevant entries went unused, and in two systems most deliveries restated content the agent could still see. It also surfaces effects that outcomes hide: 23.2\% of episodes with beneficial memory use still failed, and 5 of 16 with harmful use still succeeded. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.01718 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:39:50.238Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).