Metraipolis › archive › dc_6bd9bcd43c43

Benchmarking language models and agents for scientific reasoning in evolving materials laboratories — arXiv:2611.12014

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.12014
License (as archived)
CC0-1.0
Archived
2026-11-10T18:41:41.383Z
SHA-256 of the archived text
21e842a23dfa7f590acec472a4eb132396f902a022e2ad6e269d75e2604721b8
Stable URL of this page
https://metraipolis.com/documento/dc_6bd9bcd43c43

How to cite

Benchmarking language models and agents for scientific reasoning in evolving materials laboratories — arXiv:2611.12014 — arXiv (en), https://arxiv.org/abs/2611.12014 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6bd9bcd43c43 (Metraipolis archive dc_6bd9bcd43c43, sha256 21e842a23dfa7f590acec472a4eb132396f902a022e2ad6e269d75e2604721b8)
BibTeX
@misc{dc6bd9bcd43c43,
  title = {Benchmarking language models and agents for scientific reasoning in evolving materials laboratories — arXiv:2611.12014},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.12014},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6bd9bcd43c43; sha256 21e842a23dfa7f590acec472a4eb132396f902a022e2ad6e269d75e2604721b8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.12014. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models and the multi-agent systems built around them could accelerate materials discovery by connecting scientific knowledge with experimental decisions. Existing benchmarks establish capabilities in domain knowledge, multimodal interpretation and scientific tool use. As language models increasingly support materials research workflows, their ability to distinguish closely related material states and reason accurately with quantitative evidence remains insufficiently evaluated. These abilities are essential for comparing material responses, assessing mechanisms and maintaining reliable judgements as experimental evidence accumulates. We introduce MatScope to evaluate three connected capabilities. M1 tests unaided, condition-specific reasoning through 200 tasks spanning ten research functions, using aluminium and high-entropy alloys as representative material systems. M2 tests agent-assisted transfer of related knowledge to unresolved materials questions. MatEvoBench-20 tests initial forecasts across scientific dependency chains and revisions of still-hidden responses as observations accumulate. Across eleven model deployments, including GPT-6 Astra and Claude Fable 5.1, given-input calculations and many physical explanations are handled well. Fine-grained response estimates, comparisons between similar states and quantitatively supported choices remain substantially weaker. Related literature improves selected estimates, while condition and baseline mismatches limit transfer. Progressive observations calibrate some downstream responses and destabilize earlier accurate predictions. MatScope establishes measurable targets for developing reliable scientific reasoning in future self-evolving materials laboratories and materials-aware world models.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Materials Science (cond-mat.mtrl-sci); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.12014 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.532Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).