Metraipolis › archive › dc_ea413635b115

PSGAgent: Enhancing MLLMs' Dynamic Visual Perception for Panoptic Scene Graph Generation via Agentic Reasoning — arXiv:2611.07408

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.07408
License (as archived): CC0-1.0
Archived: 2026-11-06T14:42:21.855Z
SHA-256 of the archived text: 86463d6185011f7eba4baa92b4f7089e32e664f3e76273d5a9f5f15def118cd6
Stable URL of this page: https://metraipolis.com/documento/dc_ea413635b115

How to cite

PSGAgent: Enhancing MLLMs' Dynamic Visual Perception for Panoptic Scene Graph Generation via Agentic Reasoning — arXiv:2611.07408 — arXiv (en), https://arxiv.org/abs/2611.07408 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea413635b115 (Metraipolis archive dc_ea413635b115, sha256 86463d6185011f7eba4baa92b4f7089e32e664f3e76273d5a9f5f15def118cd6)

BibTeX

@misc{dcea413635b115,
  title = {PSGAgent: Enhancing MLLMs' Dynamic Visual Perception for Panoptic Scene Graph Generation via Agentic Reasoning — arXiv:2611.07408},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.07408},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea413635b115; sha256 86463d6185011f7eba4baa92b4f7089e32e664f3e76273d5a9f5f15def118cd6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07408. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Panoptic Scene Graph generation (PSG) requires holistic yet fine-grained reasoning over complex relations among objects. However, current Multimodal Large Language Models (MLLMs) applied to PSG suffer from a perception bottleneck: they function as static observers limited by fixed-resolution views. As a result, models often overlook critical relations involving small or peripheral objects and struggle to differentiate ambiguous predicates in complex scenes. To address these limitations, we propose PSGAgent, an agentic framework that enhances visual perception for PSG through a think-act-observe loop. PSGAgent allows MLLMs to dynamically refine their visual perception using two tools: a Region Zoom-In Tool for local exploration of small or boundary relations and a Visual Highlight Tool for relation-focused segmentation that filters out irrelevant contextual noise. To optimize decision-making, we construct a two-stage training pipeline. First, we create a high-quality, error-aware Multimodal Chain-of-Thought (M-CoT) dataset that provides robust supervision for cold-start learning. Then, we design a novel hierarchical reward mechanism integrated with Group Sequence Policy Optimization (GSPO) to enable strategic and adaptive tool usage. Extensive experiments on the PSG benchmark show that PSGAgent achieves SOTA performance, demonstrating strong capability in both holistic and fine-grained scene understanding.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.07408 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.674Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).