Metraipolis › archive › dc_aa28715afd4e

BIABench: Evaluating AI agents on real-world bioimage analysis tasks — arXiv:2609.34274

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34274
License (as archived)
CC0-1.0
Archived
2026-10-07T20:57:37.415Z
SHA-256 of the archived text
819191f15be5ba5c5417a6a5c2ec29e40dfc66ec4e334b6a3f4b2fdd0def785b
Stable URL of this page
https://metraipolis.com/documento/dc_aa28715afd4e

How to cite

BIABench: Evaluating AI agents on real-world bioimage analysis tasks — arXiv:2609.34274 — arXiv (en), https://arxiv.org/abs/2609.34274 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa28715afd4e (Metraipolis archive dc_aa28715afd4e, sha256 819191f15be5ba5c5417a6a5c2ec29e40dfc66ec4e334b6a3f4b2fdd0def785b)
BibTeX
@misc{dcaa28715afd4e,
  title = {BIABench: Evaluating AI agents on real-world bioimage analysis tasks — arXiv:2609.34274},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34274},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa28715afd4e; sha256 819191f15be5ba5c5417a6a5c2ec29e40dfc66ec4e334b6a3f4b2fdd0def785b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34274. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Artificial-intelligence (AI) agents hold promise for automating bioimage analysis, yet no benchmark evaluates whether they can carry out real-world analyses end to end. Such analyses are hard for agents because 2D images, 3D volumes and time-lapse sequences are often too large to read as context, so an agent must choose and run an analysis through code, specialized software and rendered views. Published studies make this capability testable, because each pairs raw images with a peer-reviewed result. We introduce BIABench, a benchmark of 16 tasks reconstructed from published biological studies that retain their scientific questions, imaging data and ground truth. The tasks span eleven analysis subtasks and modalities from H&E histology to single-molecule localization microscopy. Each submission receives an outcome score, which compares the output files with the ground truth using field-standard metrics, and a process score, in which a vision-language model judges method choice and quality control against an expert-written rubric. We evaluated general-purpose and biology-specific agents across several language models, with repeated runs of every task. Routine two-dimensional tasks were solved well, but on some tasks that added a third dimension or a time axis no agent scored above 0.19. Neither biological specialization, stronger models nor detailed expert instructions closed this gap. The agents were also unreliable, with scores varying more between repeated runs of one agent than between different agents, and without ground truth a correct run could not be told from a wrong one by its process score or by the time spent. Released openly with its data and code, BIABench provides a verifiable framework for evaluating, and eventually training, agents for reliable long-horizon bioimage analysis.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.34274 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.502Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).