Metraipolis › archive › dc_4afbc48cf51c

Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation — arXiv:2609.04726

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.04726
License (as archived)
CC0-1.0
Archived
2026-10-16T23:33:28.068Z
SHA-256 of the archived text
d39beb8cddfb9c6788e8e3a9de1a7f3d4346c4245624692e3d325f3bdc085a9a
Stable URL of this page
https://metraipolis.com/documento/dc_4afbc48cf51c

How to cite

Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation — arXiv:2609.04726 — arXiv (en), https://arxiv.org/abs/2609.04726 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4afbc48cf51c (Metraipolis archive dc_4afbc48cf51c, sha256 d39beb8cddfb9c6788e8e3a9de1a7f3d4346c4245624692e3d325f3bdc085a9a)
BibTeX
@misc{dc4afbc48cf51c,
  title = {Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation — arXiv:2609.04726},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.04726},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4afbc48cf51c; sha256 d39beb8cddfb9c6788e8e3a9de1a7f3d4346c4245624692e3d325f3bdc085a9a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04726. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Synthetic Aperture Radar (SAR) images have all-weather, day-and-night observation capabilities. However, compared with optical images, their speckle noise and non-intuitive scattering mechanism limit the interpretability of the images. Generative models for SAR-to-optical (S2O) conversion can improve visual interpretability, but existing methods often ignore the constraints on semantic structure, which are necessary for downstream tasks, for the sake of visual effects. We propose a unified collaborative dual-task learning framework, termed BMT (Bridging Modalities and Tasks), that jointly optimizes S2O image translation and semantic segmentation through a shared hierarchical Vision Transformer. The framework integrates: (1) a LocalViTBlock that fuses global self-attention with spatial depthwise convolution through a learnable gating mechanism; (2) an enhanced output module combining multi-scale refinement processing, color correction and anti-aliasing, which calibrates channel-level color statistics through feature fusion; (3) a ControlNet-style conditional injection mechanism that encodes SAR wavelet features and segmentation labels into a multi-scale feature pyramid and injects them at each encoder layer through zero-initialized convolution; (4) a bounded Kendall uncertainty weighting scheme that prevents either task from dominating the shared representation. We evaluate the framework under both paired and unpaired translation settings, on the public WHU-OPT-SAR paired dataset and a self-constructed unpaired ship dataset built from HRSID and DIOR, respectively. The experimental results show that the proposed method achieves competitive S2O translation quality and semantic segmentation performance. The dataset and source code have been publicly released at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-07 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.04726 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T23:16:11.400Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).