Metraipolis › archive › dc_d7e2f6c57780

MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows — arXiv:2608.02642

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02642
License (as archived)
CC0-1.0
Archived
2026-10-31T12:17:29.502Z
SHA-256 of the archived text
c519c0a308a3ed11080cbf4c8c072e159b64d17b97c6adfbbf7efbd4fce1bd1c
Stable URL of this page
https://metraipolis.com/documento/dc_d7e2f6c57780

How to cite

MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows — arXiv:2608.02642 — arXiv (en), https://arxiv.org/abs/2608.02642 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d7e2f6c57780 (Metraipolis archive dc_d7e2f6c57780, sha256 c519c0a308a3ed11080cbf4c8c072e159b64d17b97c6adfbbf7efbd4fce1bd1c)
BibTeX
@misc{dcd7e2f6c57780,
  title = {MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows — arXiv:2608.02642},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02642},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d7e2f6c57780; sha256 c519c0a308a3ed11080cbf4c8c072e159b64d17b97c6adfbbf7efbd4fce1bd1c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02642. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Accelerating scientific discovery is among the most consequential applications of AI, and computational biomolecular simulation stands out as a particularly promising target within this broader effort. Coding agents promise to automate significant portions of this workflow, yet their reliability on realistic molecular dynamics (MD) tasks remains poorly characterized. To address this issue, we introduce MDArena, a benchmark of 50 containerized tasks drawn from active biomolecular simulation projects, spanning 29 molecular systems and 14 broad research protocols, including trajectory analysis, complex system preparation, free-energy protocols, and enhanced sampling. We evaluate six model/harness configurations spanning Codex and OpenCode. Among the evaluated configurations, Codex GPT-5.5 at extra-high reasoning effort performs best, reaching 24/50 Strict-Pass@1 successes (48%), followed by Codex GPT-5.5 Medium with 21/50, and OpenCode Gemini Flash 3.5 with 20/50. Average correctness and process rewards are substantially higher than strict success rates across all configurations, indicating that agents frequently make meaningful partial progress but fail on the fine-grained details required for reproducible scientific workflows. Hard tasks remain largely unsolved, particularly membrane-protein system preparation and alchemical free-energy setup, both unsolved or near-unsolved by every evaluated configuration. MDArena thus exposes a substantial gap between the usefulness of coding agents as supervised assistants and their reliability as autonomous MD researchers, while providing a reproducible and extensible platform for tracking progress toward closing it.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.AI).
Categorie arXiv: Chemical Physics (physics.chem-ph) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.02642 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:05:52.904Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).