Metraipolis › archive › dc_1475fd656ec8

Evaluating a 4B open-weights local LLM for agentic DFT workflows: a literature reproducibility audit — arXiv:2608.29665

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29665
License (as archived)
CC0-1.0
Archived
2026-10-17T08:59:40.484Z
SHA-256 of the archived text
d1d36ae9c4e0d01975fb86e8ac46254fe933a162b5ef53466547506a9a60efe2
Stable URL of this page
https://metraipolis.com/documento/dc_1475fd656ec8

How to cite

Evaluating a 4B open-weights local LLM for agentic DFT workflows: a literature reproducibility audit — arXiv:2608.29665 — arXiv (en), https://arxiv.org/abs/2608.29665 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1475fd656ec8 (Metraipolis archive dc_1475fd656ec8, sha256 d1d36ae9c4e0d01975fb86e8ac46254fe933a162b5ef53466547506a9a60efe2)
BibTeX
@misc{dc1475fd656ec8,
  title = {Evaluating a 4B open-weights local LLM for agentic DFT workflows: a literature reproducibility audit — arXiv:2608.29665},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29665},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1475fd656ec8; sha256 d1d36ae9c4e0d01975fb86e8ac46254fe933a162b5ef53466547506a9a60efe2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29665. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agentic workflows in materials science relying on hosted commercial models face severe reproducibility, economic, and data-privacy constraints. To explore fully local agentic science, this work evaluates an open-weights Qwen3:4B model executing an autonomous scientific pipeline across varying hardware constraints. Applied to pentagonal two-dimensional materials, the system extracts parameters from unstructured text, translates them into density functional theory (DFT) inputs, and drives simulations to convergence under a strict neurosymbolic architecture where agents propose and deterministic code disposes. The workflow is guarded by verbatim text grounding and multi-pass inference unions to counteract hardware-induced structural collapse. Evaluated against 201 expert judgements, the extractor achieves 95.7% precision (95% CI 90.3-98.1%) and 67.3% recall (59.8-74.0%), ensuring extracted parameters are strictly factual. However, precision identifying absent parameters does not exceed 47.0%, establishing that the measured omission rate constitutes a loose upper bound on true literature incompleteness. Across three hardware configurations, complete GPU residency governs extraction quality more fundamentally than weight or cache precision, raising Matthews correlation from 0.414 to 0.530 at fixed quantisation and to 0.560 with an unquantised cache. A corpus-scale audit indicates only 19 (33.3%) of the 57 studies are reproducible in principle, reporting every method parameter needed to re-initialise the calculation. Driven to convergence, the workflow reproduces published lattice constants with a mean absolute relative error of 2.3% where the relaxed structure retains its prototype, establishing that lightweight open-weights models can reliably drive autonomous agentic workflows when bounded by deterministic code gates.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-01 in cs.SE).
Categorie arXiv: Materials Science (cond-mat.mtrl-sci) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.29665 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T08:37:02.099Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).