Metraipolis › archive › dc_f606861201eb
Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness? — arXiv:2610.01616
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.01616
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T12:14:30.034Z
- SHA-256 of the archived text
277298c8ee4661812483bb7c899efe419e5086d0ec1f167da1ece801afde58c3- Stable URL of this page
https://metraipolis.com/documento/dc_f606861201eb
How to cite
Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness? — arXiv:2610.01616 — arXiv (en), https://arxiv.org/abs/2610.01616 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f606861201eb (Metraipolis archive dc_f606861201eb, sha256 277298c8ee4661812483bb7c899efe419e5086d0ec1f167da1ece801afde58c3)
BibTeX
@misc{dcf606861201eb,
title = {Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness? — arXiv:2610.01616},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.01616},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f606861201eb; sha256 277298c8ee4661812483bb7c899efe419e5086d0ec1f167da1ece801afde58c3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01616. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The emergence of foundation models for molecular property prediction requires a high degree of AI data readiness, including reliable metadata annotation. However, both public repositories and industrial screening databases suffer from missing, inconsistent, or conflated assay annotations. In this work, we quantify the extent of missing annotations in PubChem for the BioAssay Ontology (BAO) assay format and physical detection method fields and investigate whether open-source and proprietary large language models (LLMs) can reliably predict and audit metadata annotations directly from the assay text. In our assessment, we found that the annotation coverage across PubChem's $\sim$2 million bioassays is critically sparse, 36\% lacking an assay format, 89\% a BioAssay type, and >99.9\% any BAO-mapped assay format or detection technology term. This motivates the need for automated test-metadata curation. Using evaluation sets derived from PubChem and ChEMBL, we assess the agreement of seven open-source and proprietary LLMs with existing silver labels. Recall is at least 0.96 for biochemical and cell-based assay formats, with a similar pattern for detection technology, although disagreements increase on under-represented classes. Manual inspection shows that many of these disagreements trace back to inconsistencies between silver sources rather than to LLM error. Moreover, in a qualitative study with a senior industrial curator, LLM-generated evidence prompted the expert to revise some of their own labels, showing LLMs can flag potentially mislabeled assays. Across the study, performance differences between proprietary and open-source models were small. Together, these results suggest LLMs can support the large-scale annotation and auditing of assay metadata, though per-class reliability estimates and targeted human review remain necessary before such labels enter downstream ML pipelines. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Databases (cs.DB); Quantitative Methods (q-bio.QM). Pagina del lavoro: https://arxiv.org/abs/2610.01616 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T10:21:28.202Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).