Metraipolis › archive › dc_4bc2aa2793c5
DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents — arXiv:2608.20664
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.20664
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T10:34:35.071Z
- SHA-256 of the archived text
0f4d5fdaf6297fb3a5b4f90567d63b1616f38a01ef28fe0c2e29ca636b2fa8e9- Stable URL of this page
https://metraipolis.com/documento/dc_4bc2aa2793c5
How to cite
DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents — arXiv:2608.20664 — arXiv (en), https://arxiv.org/abs/2608.20664 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4bc2aa2793c5 (Metraipolis archive dc_4bc2aa2793c5, sha256 0f4d5fdaf6297fb3a5b4f90567d63b1616f38a01ef28fe0c2e29ca636b2fa8e9)
BibTeX
@misc{dc4bc2aa2793c5,
title = {DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents — arXiv:2608.20664},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.20664},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4bc2aa2793c5; sha256 0f4d5fdaf6297fb3a5b4f90567d63b1616f38a01ef28fe0c2e29ca636b2fa8e9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20664. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
DreamBench-SWE is a multi-session benchmark for software-agent memory hygiene in which later software tasks depend on non-inferable evidence from earlier sessions and are scored by executable hidden oracles. We report the original scaled v2 fold and a separately preregistered v2.1 successor audit designed after that study but frozen before successor outcome inspection. The successor run completed 360/360 work units and 720/720 S3 cells across four conditions. In the original fold, the primary DF-hybrid--B5 contrast was null (95/180 versus 89/180; clustered p=.518, Holm p=1), not evidence of equivalence, and C9/C10 retained B0-headroom limitations. In the successor, no external memory achieved 21/180 passes (rate 0.1167), deterministic verbatim event memory 82/180 (rate 0.4556), the typed-plus-raw reference probe 83/180 (rate 0.4611), and one pinned hosted Mem0 literal-storage configuration 97/180 (rate 0.5389). The registered six-slot Family A retained unavailable slots at p=1; all three available comparisons against no memory rejected after Holm correction. Both preregistered mechanism contrasts were unavailable after pre-evaluation conformance rejection. The secondary literal-storage-versus-verbatim comparison was nonconfirmatory and sensitivity-dependent, while the comparison with the reference probe did not reject. The audit therefore supports DreamBench-SWE as a discriminating executable profile benchmark and characterizes one exact hosted-memory configuration, but it does not establish an external-system mechanism, superiority among memory-bearing conditions, equivalence, or broad product generality. The original v2.0.5 findings and artifacts remain unchanged. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-24 in cs.SE). Categorie arXiv: Artificial Intelligence (cs.AI) ; Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2608.20664 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T10:28:20.999Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).