Metraipolis › archive › dc_b992af2ab19e

Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01718
License (as archived)
CC0-1.0
Archived
2026-11-03T10:22:20.428Z
SHA-256 of the archived text
a4eb662202cefa14f0b4d64bfe878196e7e5eb34664c51597a0854f304de0432
Stable URL of this page
https://metraipolis.com/documento/dc_b992af2ab19e

How to cite

Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718 — arXiv (en), https://arxiv.org/abs/2611.01718 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b992af2ab19e (Metraipolis archive dc_b992af2ab19e, sha256 a4eb662202cefa14f0b4d64bfe878196e7e5eb34664c51597a0854f304de0432)
BibTeX
@misc{dcb992af2ab19e,
  title = {Delivered, Not Used: Evaluating Agent Memory at Its Point of Use — arXiv:2611.01718},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01718},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b992af2ab19e; sha256 a4eb662202cefa14f0b4d64bfe878196e7e5eb34664c51597a0854f304de0432},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01718. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Task success, retrieval quality, and cost do not show whether an LLM agent used a delivered memory or what followed from that use. We introduce MemUse, which evaluates memory at its point of use in recorded trajectories: each memory delivery is compared with what the agent could already see and followed through the agent's later actions and their consequences. An agentic judge rates each delivered entry's relevance, accuracy, novelty, influence, and observed effect; programmatic checks enforce complete reading of cited evidence, and fixed rules aggregate the ratings into verdicts linked to memory histories and delivery cost. On 8,000 SWE-bench Verified episodes from three memory systems, resolve rates stay within 0.4 percentage points of a no-memory baseline, yet MemUse shows that most delivered entries were irrelevant, most relevant entries went unused, and in two systems most deliveries restated content the agent could still see. It also surfaces effects that outcomes hide: 23.2\% of episodes with beneficial memory use still failed, and 5 of 16 with harmful use still succeeded.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.01718 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:19:49.832Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).