Metraipolis › archive › dc_af37441d09f2

CMA-OT: Hierarchical Expert Supervision for Dance-to-Music Generation — arXiv:2609.13118

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.13118
License (as archived)
CC0-1.0
Archived
2026-10-16T03:03:01.528Z
SHA-256 of the archived text
e1a79ee7df5f99b1265cae84270ab8aa1d42a89e9b81846fefca604c2d85784d
Stable URL of this page
https://metraipolis.com/documento/dc_af37441d09f2

How to cite

CMA-OT: Hierarchical Expert Supervision for Dance-to-Music Generation — arXiv:2609.13118 — arXiv (en), https://arxiv.org/abs/2609.13118 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_af37441d09f2 (Metraipolis archive dc_af37441d09f2, sha256 e1a79ee7df5f99b1265cae84270ab8aa1d42a89e9b81846fefca604c2d85784d)
BibTeX
@misc{dcaf37441d09f2,
  title = {CMA-OT: Hierarchical Expert Supervision for Dance-to-Music Generation — arXiv:2609.13118},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.13118},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_af37441d09f2; sha256 e1a79ee7df5f99b1265cae84270ab8aa1d42a89e9b81846fefca604c2d85784d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13118. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dance-to-music (D2M) generation aims to synthesize music that is rhythmically and stylistically aligned with dance videos. A key challenge arises from the semantic mismatch between sparse dance cues, such as rhythm and style, and the dense information required for music composition, including structure, instrumentation, and expressive dynamics. Existing methods typically rely on these sparse cues and supervise only the final audio output, resulting in poorly learned music representations and generated music with limited musicality and structural coherence. To address these issues, we propose Curriculum-guided Multi-scale representation Alignment with scale-aware Optimal Transport (CMA-OT), a novel paradigm that leverages an external music expert to provide hierarchical supervision for the generator's latent features, bridging the semantic gap and enhancing representation learning. To effectively incorporate hierarchical supervision, we introduce a curriculum-guided multi-scale learning strategy that progressively transfers musical knowledge from the expert to the music generator, enabling stable and effective representation learning. Moreover, to accommodate the semantic and structural variations across different expert scales and achieve fine-grained alignment under temporal mismatch, we propose a scale-aware optimal transport alignment mechanism, which models soft correspondences between hierarchical expert representations and the generator's latent features. Extensive experiments on two datasets demonstrate that CMA-OT achieves state-of-the-art performance in rhythmic synchronization, perceptual quality, and overall music generation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2609.13118 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T02:51:27.836Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).