Metraipolis › archive › dc_5dbdbef56471

SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation — arXiv:2608.08147

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08147
License (as archived)
CC0-1.0
Archived
2026-10-19T09:15:39.182Z
SHA-256 of the archived text
4084cb6919abca8e47e8654a83fc82b0362f7a935e60719c3a3fa024edb3345c
Stable URL of this page
https://metraipolis.com/documento/dc_5dbdbef56471

How to cite

SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation — arXiv:2608.08147 — arXiv (en), https://arxiv.org/abs/2608.08147 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5dbdbef56471 (Metraipolis archive dc_5dbdbef56471, sha256 4084cb6919abca8e47e8654a83fc82b0362f7a935e60719c3a3fa024edb3345c)
BibTeX
@misc{dc5dbdbef56471,
  title = {SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation — arXiv:2608.08147},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08147},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5dbdbef56471; sha256 4084cb6919abca8e47e8654a83fc82b0362f7a935e60719c3a3fa024edb3345c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08147. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Audio-Visual Instance Segmentation (AVIS) aims to simultaneously classify, segment, and track sounding objects within video sequences. Unlike Audio-Visual Semantic Segmentation (AVS), AVIS involves instance-level modeling across longer video sequences, introducing two key challenges: (1) complex modality-state changes disrupt long-range modeling, and (2) substantial structural and distributional discrepancies between modalities hinder precise instance-level association. Existing methods rely on fixed-step Transformers and recursive Mamba models, lacking adaptability to modality-state changes. In addition, methods performing implicit matching ignore the inherent distributional inconsistencies. To address these issues, we propose a framework with Adaptive Dynamic Step Modulation (ADSM) and Optimal Transport-based Matching Modulation (OT-MM). ADSM adaptively modulates Mamba step sizes using temporal variation, cross-modal discrepancy, and historical context, balancing rapid response to modality-state changes with stable long-range modeling. OT-MM explicitly formulates instance-level cross-modal matching as an entropy-regularized optimal transport problem solved via log-domain Sinkhorn iterations, and further enforces distribution-level coherence with an MMD regularizer. Extensive experiments demonstrate state-of-the-art performance on the AVIS benchmark (+3.76 FSLA, +2.75 HOTA, +2.58 mAP), verified through comprehensive qualitative visualizations. The code and model are available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.MM).
Categorie arXiv: Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2608.08147 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:13:27.072Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).