Metraipolis › archive › dc_be7b04bada6f

Vision-Language Agents for Active Perception in Optics Laboratories — arXiv:2609.32918

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32918
License (as archived)
CC0-1.0
Archived
2026-10-07T21:00:47.939Z
SHA-256 of the archived text
e47d436a8aa7faaaebe07878599b0c8579619b70d5f6b196f94d88c29e3d2427
Stable URL of this page
https://metraipolis.com/documento/dc_be7b04bada6f

How to cite

Vision-Language Agents for Active Perception in Optics Laboratories — arXiv:2609.32918 — arXiv (en), https://arxiv.org/abs/2609.32918 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_be7b04bada6f (Metraipolis archive dc_be7b04bada6f, sha256 e47d436a8aa7faaaebe07878599b0c8579619b70d5f6b196f94d88c29e3d2427)
BibTeX
@misc{dcbe7b04bada6f,
  title = {Vision-Language Agents for Active Perception in Optics Laboratories — arXiv:2609.32918},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32918},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_be7b04bada6f; sha256 e47d436a8aa7faaaebe07878599b0c8579619b70d5f6b196f94d88c29e3d2427},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32918. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Vision-language models (VLMs) are increasingly being used in scientific workflows, but their ability as agents to directly control laboratory experiments from visual feedback remains underexplored. This capability is important because many laboratory tasks do not naturally provide dense, pre-defined numerical objectives: informative signals can be sparse, intermittent, or visually ambiguous. A more general laboratory agent should instead be able to interpret visual observations, take actions to acquire useful feedback, and adapt its behavior based on the consequences of those actions. We study whether general-purpose VLMs can perform this kind of closed-loop scientific control using experimental optics as a testbed. We evaluate agents on three experimental systems that isolate distinct capabilities: a Michelson interferometer, a two-mirror cavity, and a four-mirror optical relay. The agents observe camera images, directly issue actuator and measurement commands, and retain their interaction history without receiving an engineered scalar objective during control. Across these experiments and matched simulations, we find that, given task-specific natural-language guidance, VLMs can estimate actuator-response relationships, resolve ambiguous observations through intervention, and actively create informative visual feedback when signals are sparse. These results suggest that pretrained multimodal models can serve as important decision-making agents within the experimental loop. Our work also establishes optics as a physically grounded testbed for visual reasoning and active perception in scientific agents.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Optics (physics.optics) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.32918 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.504Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).