Metraipolis › archive › dc_5dbdbef56471
SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation — arXiv:2608.08147
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.08147
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:15:39.182Z
- SHA-256 of the archived text
4084cb6919abca8e47e8654a83fc82b0362f7a935e60719c3a3fa024edb3345c- Stable URL of this page
https://metraipolis.com/documento/dc_5dbdbef56471
How to cite
SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation — arXiv:2608.08147 — arXiv (en), https://arxiv.org/abs/2608.08147 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5dbdbef56471 (Metraipolis archive dc_5dbdbef56471, sha256 4084cb6919abca8e47e8654a83fc82b0362f7a935e60719c3a3fa024edb3345c)
BibTeX
@misc{dc5dbdbef56471,
title = {SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation — arXiv:2608.08147},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.08147},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5dbdbef56471; sha256 4084cb6919abca8e47e8654a83fc82b0362f7a935e60719c3a3fa024edb3345c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08147. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Audio-Visual Instance Segmentation (AVIS) aims to simultaneously classify, segment, and track sounding objects within video sequences. Unlike Audio-Visual Semantic Segmentation (AVS), AVIS involves instance-level modeling across longer video sequences, introducing two key challenges: (1) complex modality-state changes disrupt long-range modeling, and (2) substantial structural and distributional discrepancies between modalities hinder precise instance-level association. Existing methods rely on fixed-step Transformers and recursive Mamba models, lacking adaptability to modality-state changes. In addition, methods performing implicit matching ignore the inherent distributional inconsistencies. To address these issues, we propose a framework with Adaptive Dynamic Step Modulation (ADSM) and Optimal Transport-based Matching Modulation (OT-MM). ADSM adaptively modulates Mamba step sizes using temporal variation, cross-modal discrepancy, and historical context, balancing rapid response to modality-state changes with stable long-range modeling. OT-MM explicitly formulates instance-level cross-modal matching as an entropy-regularized optimal transport problem solved via log-domain Sinkhorn iterations, and further enforces distribution-level coherence with an MMD regularizer. Extensive experiments demonstrate state-of-the-art performance on the AVIS benchmark (+3.76 FSLA, +2.75 HOTA, +2.58 mAP), verified through comprehensive qualitative visualizations. The code and model are available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.MM). Categorie arXiv: Multimedia (cs.MM). Pagina del lavoro: https://arxiv.org/abs/2608.08147 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:13:27.072Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).