Metraipolis › archive › dc_5d19489b698e
The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory — arXiv:2610.34767
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.34767
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T07:58:11.904Z
- SHA-256 of the archived text
61afb208792d3227e74a2c8085c433f9944225642f0ae69b805d925917a5f746- Stable URL of this page
https://metraipolis.com/documento/dc_5d19489b698e
How to cite
The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory — arXiv:2610.34767 — arXiv (en), https://arxiv.org/abs/2610.34767 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5d19489b698e (Metraipolis archive dc_5d19489b698e, sha256 61afb208792d3227e74a2c8085c433f9944225642f0ae69b805d925917a5f746)
BibTeX
@misc{dc5d19489b698e,
title = {The Right Fact, the Wrong Decision: SciValidityBench for Scientific Agent Memory — arXiv:2610.34767},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.34767},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5d19489b698e; sha256 61afb208792d3227e74a2c8085c433f9944225642f0ae69b805d925917a5f746},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34767. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Most agent-memory benchmarks frame long-term memory as retrieval: can an agent recover the relevant record from a long and evolving history? Scientific agents face a stricter problem: a correctly remembered fact may still be invalid to use. The same reported test result can support confirmation when the outcome and stopping rule were fixed in advance, yet only exploration when the outcome was selected from twenty inspected results. We call this scientific use validity: whether remembered evidence is eligible to influence a decision under the state governing its interpretation. We introduce SciValidityBench, an executable benchmark factory with 64 TaskSpecs spanning five validity modules and six scientific domains. TaskSpecs couple generated experimental histories with scientific-state definitions, programmatic decision oracles, controlled interventions, and automatic scoring. The SciStateForge compiler-reader framework turns distributed histories into explicit decision-governing state. Counterfactual pairs and targeted corruption-repair controls then test whether models use those dependencies and localize failures. Across four model families, compiled scientific state improves valid decision-making by 9.2-18.8 percentage points over a generic Event Graph. Retrieval alone does not reproduce this cross-family gain, while content-identical flat and typed views show that the improvement comes from exposing governing state rather than a particular graph serialization. SciValidityBench thus shifts memory evaluation from finding the right fact to deciding whether that fact should control the decision. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.34767 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.713Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).