Metraipolis › archive › dc_bc8ef189a02f
HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion — arXiv:2608.11913
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.11913
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T05:49:34.383Z
- SHA-256 of the archived text
9e531844e077a4e17418aa5654e3b7d739d20d6f2e57ea7d9c852d78375b1b08- Stable URL of this page
https://metraipolis.com/documento/dc_bc8ef189a02f
How to cite
HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion — arXiv:2608.11913 — arXiv (en), https://arxiv.org/abs/2608.11913 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bc8ef189a02f (Metraipolis archive dc_bc8ef189a02f, sha256 9e531844e077a4e17418aa5654e3b7d739d20d6f2e57ea7d9c852d78375b1b08)
BibTeX
@misc{dcbc8ef189a02f,
title = {HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion — arXiv:2608.11913},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.11913},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bc8ef189a02f; sha256 9e531844e077a4e17418aa5654e3b7d739d20d6f2e57ea7d9c852d78375b1b08},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11913. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Video-to-audio (V2A) generation faces significant challenges in achieving precise temporal synchronization and high perceptual quality due to the complex, ambiguous relationship between visual and auditory cues. Existing methods typically compress video inputs into single feature representations, leading to significant loss of temporal dynamics and fine-grained visual information. These approaches also rely on reconstruction-based training objectives that poorly correlate with human perceptual judgments of audio quality and appropriateness. We propose HarmoniDPO, a novel framework that integrates preference-based optimization into diffusion-based V2A generation to address these limitations. (1) Our approach leverages a dual video representation: combining global context with frame-wise features to preserve temporal dynamics and semantic detail. (2) Inspired by reinforcement learning from human feedback (RLHF), HarmoniDPO employs online Direct Preference Optimization (online-DPO) to fine-tune a diffusion-based V2A model from preference judgments, enhancing perceptual quality and alignment. (3) Additionally, we introduce Dual-scale Diffusion Search (DDS), a test time scaling algorithm that adaptively optimizes output fidelity during inference. Experiments demonstrate that HarmoniDPO outperforms state-of-the-art methods in audio-video synchronization and subjective audio quality, offering a robust solution for generating realistic, human-preferred audio from video. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-13 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2608.11913 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T05:42:25.895Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).