Metraipolis › archive › dc_f8b3e9fca031
SRRM: A Retrieval-Reconstruction Benchmark for Evaluating Long-Context Memory in Small Language Models — arXiv:2611.05386
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.05386
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:18:35.580Z
- SHA-256 of the archived text
22d0ad4134a5d46a1e2f60ba01e8703dea0b6f02386e9506ea28df23a9a6885d- Stable URL of this page
https://metraipolis.com/documento/dc_f8b3e9fca031
How to cite
SRRM: A Retrieval-Reconstruction Benchmark for Evaluating Long-Context Memory in Small Language Models — arXiv:2611.05386 — arXiv (en), https://arxiv.org/abs/2611.05386 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f8b3e9fca031 (Metraipolis archive dc_f8b3e9fca031, sha256 22d0ad4134a5d46a1e2f60ba01e8703dea0b6f02386e9506ea28df23a9a6885d)
BibTeX
@misc{dcf8b3e9fca031,
title = {SRRM: A Retrieval-Reconstruction Benchmark for Evaluating Long-Context Memory in Small Language Models — arXiv:2611.05386},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.05386},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f8b3e9fca031; sha256 22d0ad4134a5d46a1e2f60ba01e8703dea0b6f02386e9506ea28df23a9a6885d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05386. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Small Language Models (SLMs) are increasingly deployed on consumer-grade hardware, yet their ability to preserve information across progressively longer contexts remains insufficiently understood. Existing benchmarks primarily evaluate retrieval-oriented tasks, providing limited insight into whether successful retrieval reflects comprehensive long-context memory preservation. To address this limitation, this paper introduces Simple Retrieval-Reconstruction Memory (SRRM), a lightweight and fully reproducible benchmark that jointly evaluates Targeted Retrieval and Global Information Reconstruction. This study also proposes the Memory Preservation Ratio (MPR), a benchmark-level metric that quantifies the relationship between retrieval and reconstruction performance. SRRM was evaluated using a synthetic dataset on three Qwen2.5 models (0.5B, 1.5B, and 3B parameters) across context loads ranging from 10 to 100 factual statements. Experimental results show that reconstruction performance deteriorated substantially faster than retrieval accuracy as contextual information increased, indicating that retrieval alone is insufficient to characterize long-context memory preservation. Among the evaluated models, Qwen2.5-1.5B achieved the highest retrieval accuracy (86.7%), reconstruction performance (45.7%), and MPR (0.53), outperforming the larger Qwen2.5-3B model (72.2%, 18.5%, and 0.26, respectively). These findings demonstrate that increasing parameter count does not necessarily improve long-context memory preservation. Overall, SRRM provides a reproducible and extensible benchmark for systematically evaluating long-context memory while revealing complementary memory behaviors that remain hidden under retrieval-only evaluation frameworks. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.05386 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:13:03.936Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).