Metraipolis › archive › dc_755f3db3ebd1

Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition — arXiv:2608.04623

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04623
License (as archived)
CC0-1.0
Archived
2026-10-31T11:48:02.669Z
SHA-256 of the archived text
54b0cbb59b8cbf9f07f78b0f07cc567c09b71f93c5b9d1a9028c5840417375d0
Stable URL of this page
https://metraipolis.com/documento/dc_755f3db3ebd1

How to cite

Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition — arXiv:2608.04623 — arXiv (en), https://arxiv.org/abs/2608.04623 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_755f3db3ebd1 (Metraipolis archive dc_755f3db3ebd1, sha256 54b0cbb59b8cbf9f07f78b0f07cc567c09b71f93c5b9d1a9028c5840417375d0)
BibTeX
@misc{dc755f3db3ebd1,
  title = {Visual Anchoring in Diffusion: Multimodal Zero-Shot Skeleton Action Recognition — arXiv:2608.04623},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04623},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_755f3db3ebd1; sha256 54b0cbb59b8cbf9f07f78b0f07cc567c09b71f93c5b9d1a9028c5840417375d0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04623. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Zero-shot Skeleton Action Recognition (ZSAR) remains ambiguous when unseen actions share similar skeleton joint dynamics but differ in objects or scene context. RGB provides these missing cues, yet existing multimodal methods typically maintain independent skeleton and RGB scoring branches and fuse their outputs. Without using unlabeled test data for adaptation or fusion calibration, a fixed fusion weight cannot capture class-pair-dependent modality reliability, while an adaptive rule lacks target-side feedback for deciding which branch should dominate. We bypass this weight-selection problem via the classify-by-generation paradigm, where each class is scored by how accurately a text-conditioned denoiser predicts the noise added to the skeleton feature. This formulation separates the progressively corrupted skeleton from fixed conditioning, allowing RGB and text to jointly condition a single class-scoring function rather than produce independent scores. We instantiate this idea as Multimodal Triplet Diffusion for Skeleton-Text Matching (TDSM-MM), augmenting a text-conditioned denoising Transformer with a non-diffused RGB condition token that serves as a stable visual anchor during skeleton data reconstruction. Our proposed TDSM-MM has been ablated via extensive experiments and achieved the best inductive accuracy on three of four NTU-60/120 splits and surpasses the transductive state-of-the-art on NTU-120 96/24 (i.e., 71.3% vs. 69.1%), without test-time adaptation, suggesting that diffusion-based methods can be a promising direction for zero-shot learning.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.04623 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:37:40.696Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).