Metraipolis › archive › dc_035cdbe89fb5

MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles — arXiv:2610.04800

Source: arXiv
Original: https://arxiv.org/abs/2610.04800
License (as archived): CC0-1.0
Archived: 2026-10-06T05:24:49.337Z
SHA-256 of the archived text: 88d5259e47418acbc9594bc3f9d826c28c062cff73075abfb9089493cdf46914
Stable URL of this page: https://metraipolis.com/documento/dc_035cdbe89fb5

How to cite

MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles — arXiv:2610.04800 — arXiv, https://arxiv.org/abs/2610.04800 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_035cdbe89fb5 (Metraipolis archive dc_035cdbe89fb5, sha256 88d5259e47418acbc9594bc3f9d826c28c062cff73075abfb9089493cdf46914)

BibTeX

@misc{dc035cdbe89fb5,
  title = {MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles — arXiv:2610.04800},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04800},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_035cdbe89fb5; sha256 88d5259e47418acbc9594bc3f9d826c28c062cff73075abfb9089493cdf46914},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04800. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Graphical consoles offer a practical interface for medical acquisition assistance, allowing agents to work through the controls and visual feedback used by human operators. Reliable assistance requires linking on-screen anatomy to acquisition decisions that determine what image evidence becomes available next. We introduce MedImageOSWorld, a benchmark for evaluating this capability in simulated CT, MR, and ultrasound consoles. Using screenshots and mouse-and-keyboard actions, agents configure protocols, plan acquisitions, inspect the resulting images, and make corrective adjustments across seven capability levels, from console operation to feedback-driven control. Evaluation combines task-specific workflow checks with hidden anatomical ground truth to assess procedural completion and acquisition outcomes separately. A common evaluation protocol specifies episode conditions and interaction budgets, while recorded trajectories support analysis of how agents observe, act, and respond to acquisition feedback. Across eleven open-weight agents, success rates range from 3.0 to 25.0 on a 0-100 scale while workflow-progress rates reach 21.5-74.5: agents complete much of the console workflow but rarely acquire the intended anatomy. Success collapses between perception and planning, from 79-90% at the lowest three levels for the best agent to at most 12% for millimetre-level planning and 0% for closed-loop control among open-weight agents. Two proprietary agents reach success rates of about 34 and exceed the best open-weight agent mainly in acquisition quality (61 versus 42). MedImageOSWorld provides a controlled setting for studying whether general-purpose GUI agents can translate visual observations into effective medical acquisition decisions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.04800 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.521Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).