Metraipolis › archive › dc_57dbfbb6cb52

FREA: A Multi-Source Expert Benchmark for Reaction Feasibility Verification — arXiv:2610.06614

Source: arXiv
Original: https://arxiv.org/abs/2610.06614
License (as archived): CC0-1.0
Archived: 2026-10-06T05:25:41.489Z
SHA-256 of the archived text: 468557090b9f8224c523c0cfa80132308ccb8dc3b735f08b3a4fb8de20fae1ba
Stable URL of this page: https://metraipolis.com/documento/dc_57dbfbb6cb52

How to cite

FREA: A Multi-Source Expert Benchmark for Reaction Feasibility Verification — arXiv:2610.06614 — arXiv, https://arxiv.org/abs/2610.06614 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_57dbfbb6cb52 (Metraipolis archive dc_57dbfbb6cb52, sha256 468557090b9f8224c523c0cfa80132308ccb8dc3b735f08b3a4fb8de20fae1ba)

BibTeX

@misc{dc57dbfbb6cb52,
  title = {FREA: A Multi-Source Expert Benchmark for Reaction Feasibility Verification — arXiv:2610.06614},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.06614},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_57dbfbb6cb52; sha256 468557090b9f8224c523c0cfa80132308ccb8dc3b735f08b3a4fb8de20fae1ba},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.06614. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As generative models and AI agents propose chemical reactions at a scale beyond expert review, feasibility verifiers decide which proposals enter synthesis planning. But do their decisions agree with chemists across different kinds of candidates? We introduce FREA, a benchmark of 751 reactions labeled by expert chemists under an explicit feasibility criterion, drawn from retrosynthesis model proposals, zero-yield experimental records, edits by large language models (LLMs), and five negative candidate generation methods. Our evaluation finds that no verifier leads across all sources: LLMs given only the criterion are competitive with dedicated verifiers, while forward models perform best on retrosynthesis proposals but reject most feasible edits of recorded reactions at the evaluated operating points. Looking beyond aggregate scores, both forward models perform below chance when separating infeasible alternative disconnections from feasible generated candidates. To study whether negative supervision addresses these weaknesses, we also release a corpus of over 14 million recorded reactions and generated negative candidates. In matched training comparisons, adding a mixture of generated negatives to forward training raises mean AUROC across sources, but these gains do not extend to retrosynthesis proposals. Varying the generation method further shows that the largest gain on generated candidates coincides with worse proposal screening. These findings motivate evaluating verifiers against experts across sources and designing negatives for transfer to model proposals.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.06614 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.703Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).