Metraipolis › archive › dc_044d0f79823f
LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture — arXiv:2609.16730
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.16730
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T18:32:11.257Z
- SHA-256 of the archived text
cfc9fe7ab58bbb805e062d7af4e04139843e6f7dc2b5b4c072047329345fffe9- Stable URL of this page
https://metraipolis.com/documento/dc_044d0f79823f
How to cite
LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture — arXiv:2609.16730 — arXiv (en), https://arxiv.org/abs/2609.16730 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_044d0f79823f (Metraipolis archive dc_044d0f79823f, sha256 cfc9fe7ab58bbb805e062d7af4e04139843e6f7dc2b5b4c072047329345fffe9)
BibTeX
@misc{dc044d0f79823f,
title = {LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture — arXiv:2609.16730},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.16730},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_044d0f79823f; sha256 cfc9fe7ab58bbb805e062d7af4e04139843e6f7dc2b5b4c072047329345fffe9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16730. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Conversational memory changes during use, so endpoint question answering alone cannot establish how a persistent state accumulates, ages, or incorporates revisions. We introduce LSREP, a Longitudinal State-Replay Evaluation Protocol combining ordered replay, explicit lifecycle schedules, repeated probes, evolving reference answers, and mechanism-fidelity checks. Its architectural case study is ICE v2, a local-first memory middleware with typed stores, retrieval fusion, and dynamic context budgets. The private, single-user instantiation contains 1,985 turns, 219 distinct probes, and 1,211 probe-checkpoint observations across 52 checkpoints. On three ordinary-density datasets, ICE v2 has a near-zero mean quality difference from vector-RAG while selecting 32% fewer fragments but using 6.6% more estimated prompt tokens. A fourth, dense dataset exposes catastrophic failures of the unbudgeted baseline. The fidelity audit limits attribution: procedural retrieval is defective, several mechanisms are unexercised, and graph utility is not established. In a complementary matched public diagnostic, ICE v2 loses decisively to pure vector-RAG on LongMemEval: 50.8% versus 72.8% in the evidence-only oracle and 43.0% versus 69.5% in full-S. Paired differences are -22.0 points (95% CI [-26.6, -17.4]) and -26.5 ([-31.3, -21.8]). Conservative abstention accompanies severe multi-session and temporal failures. ICE uses less context in this diagnostic, establishing a quality-cost trade-off rather than superior efficiency. Together, replay, fidelity auditing, and public endpoint testing expose distinct failure modes that neither architectural descriptions nor aggregate scores identify alone. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2609.16730 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T18:03:09.462Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).