Metraipolis › archive › dc_64c1abc5eff5

ContextWeave: A Real-World Workflow Benchmark — arXiv:2608.04830

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04830
License (as archived)
CC0-1.0
Archived
2026-10-31T11:12:43.240Z
SHA-256 of the archived text
a0b634b9c459ed374505861d81599fe6dd1a2eea6c2940823e8aa6bbc7d28a89
Stable URL of this page
https://metraipolis.com/documento/dc_64c1abc5eff5

How to cite

ContextWeave: A Real-World Workflow Benchmark — arXiv:2608.04830 — arXiv (en), https://arxiv.org/abs/2608.04830 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64c1abc5eff5 (Metraipolis archive dc_64c1abc5eff5, sha256 a0b634b9c459ed374505861d81599fe6dd1a2eea6c2940823e8aa6bbc7d28a89)
BibTeX
@misc{dc64c1abc5eff5,
  title = {ContextWeave: A Real-World Workflow Benchmark — arXiv:2608.04830},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04830},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64c1abc5eff5; sha256 a0b634b9c459ed374505861d81599fe6dd1a2eea6c2940823e8aa6bbc7d28a89},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04830. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Memory is essential as language agents move from isolated tasks to long-horizon, stateful workflows, yet existing evaluations often reduce it to retrieval or question answering. We introduce ContextWeave, a longitudinal benchmark that evaluates whether recalled experience improves downstream agent performance in realistic office-work streams. ContextWeave reconstructs privacy-preserved, multi-month workflows of 14 participants into 1,005 executable tasks, including 568 core evaluation tasks, with instructions, containerized environments, trajectories, and task-specific rubrics. It measures workspace quality and alignment with participant-specific preferences, complemented by diagnostics of relevance, continuity, solvability, and robustness to misleading recall. Across six memory components under a fixed model, the strongest configuration raises Workspace Score from 68.08 to 78.20 and Preference Score from 41.50 to 70.60. With a fixed memory component, recall improves both outcomes for all five tested base models, although gains vary substantially. Our analysis shows that actionable, experience-rich memory supports workflow continuation and reduces redundant exploration more effectively than compact summaries, while it can also be more susceptible to misleading recall. These findings motivate memory systems that optimize not only retrieval relevance but also reliable use during execution.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.04830 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:02:13.431Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).