Metraipolis › archive › dc_64c1abc5eff5
ContextWeave: A Real-World Workflow Benchmark — arXiv:2608.04830
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04830
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:12:43.240Z
- SHA-256 of the archived text
a0b634b9c459ed374505861d81599fe6dd1a2eea6c2940823e8aa6bbc7d28a89- Stable URL of this page
https://metraipolis.com/documento/dc_64c1abc5eff5
How to cite
ContextWeave: A Real-World Workflow Benchmark — arXiv:2608.04830 — arXiv (en), https://arxiv.org/abs/2608.04830 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64c1abc5eff5 (Metraipolis archive dc_64c1abc5eff5, sha256 a0b634b9c459ed374505861d81599fe6dd1a2eea6c2940823e8aa6bbc7d28a89)
BibTeX
@misc{dc64c1abc5eff5,
title = {ContextWeave: A Real-World Workflow Benchmark — arXiv:2608.04830},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04830},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64c1abc5eff5; sha256 a0b634b9c459ed374505861d81599fe6dd1a2eea6c2940823e8aa6bbc7d28a89},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04830. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Memory is essential as language agents move from isolated tasks to long-horizon, stateful workflows, yet existing evaluations often reduce it to retrieval or question answering. We introduce ContextWeave, a longitudinal benchmark that evaluates whether recalled experience improves downstream agent performance in realistic office-work streams. ContextWeave reconstructs privacy-preserved, multi-month workflows of 14 participants into 1,005 executable tasks, including 568 core evaluation tasks, with instructions, containerized environments, trajectories, and task-specific rubrics. It measures workspace quality and alignment with participant-specific preferences, complemented by diagnostics of relevance, continuity, solvability, and robustness to misleading recall. Across six memory components under a fixed model, the strongest configuration raises Workspace Score from 68.08 to 78.20 and Preference Score from 41.50 to 70.60. With a fixed memory component, recall improves both outcomes for all five tested base models, although gains vary substantially. Our analysis shows that actionable, experience-rich memory supports workflow continuation and reduces redundant exploration more effectively than compact summaries, while it can also be more susceptible to misleading recall. These findings motivate memory systems that optimize not only retrieval relevance but also reliable use during execution. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.04830 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:02:13.431Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).