Metraipolis › archive › dc_af3bb67b8b63
Intervention, Not Shared Latents: Blocking Visual Shortcuts in Audio-Video Generation — arXiv:2609.22361
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22361
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T07:06:27.894Z
- SHA-256 of the archived text
9bca3e235e69be600dcda5b8c4ef36c2699e2e545a7fef4fbeaa4b5b95113d89- Stable URL of this page
https://metraipolis.com/documento/dc_af3bb67b8b63
How to cite
Intervention, Not Shared Latents: Blocking Visual Shortcuts in Audio-Video Generation — arXiv:2609.22361 — arXiv (en), https://arxiv.org/abs/2609.22361 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_af3bb67b8b63 (Metraipolis archive dc_af3bb67b8b63, sha256 9bca3e235e69be600dcda5b8c4ef36c2699e2e545a7fef4fbeaa4b5b95113d89)
BibTeX
@misc{dcaf3bb67b8b63,
title = {Intervention, Not Shared Latents: Blocking Visual Shortcuts in Audio-Video Generation — arXiv:2609.22361},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22361},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_af3bb67b8b63; sha256 9bca3e235e69be600dcda5b8c4ef36c2699e2e545a7fef4fbeaa4b5b95113d89},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22361. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Joint audio--video (AV) generators are trained on data in which \emph{what an event looks like} and \emph{what it sounds like} are spuriously correlated. We present a \emph{controlled causal study} of the resulting failure mode. In an AV structural causal model where the audio is, by construction, independent of the video's nuisance appearance, models that let audio read video directly---through cross-attention or a shared latent---learn a \emph{visual shortcut}: they predict sound from appearance rather than the causal event and, when the appearance--event correlation is broken at test time, synthesize the wrong event's sound. Crucially, the popular remedy of routing both modalities through a \emph{shared common-cause latent} does \emph{not} fix this---a bottleneck, an unsupervised shared/private factorization, and a faithful shared-prior model all grab the appearance proxy and fail like the direct model. Blocking the shortcut instead requires an \emph{intervention on the nuisance}: under the stated assumptions we prove that counterfactual invariance is necessary and sufficient to identify the causal predictor, and we verify the mechanism from feature-vector SCMs to procedural pixel video, real images with spectrogram audio, moving real digits, and a conditional generator. On a \emph{real, pretrained} V2A generator (MMAudio), an input-intervention test shows the model is far from invariant to sound-irrelevant edits, though a generic-noise control reveals it is broadly input-brittle rather than specifically colour-shortcutting---clean isolation of the shortcut needs the controlled confounds our synthetic studies provide. We characterize \emph{when} the shortcut occurs, compare the objective against supervised counterfactual augmentation, and isolate the \emph{unknown-nuisance} regime---where the intervention cannot be applied---as the central open problem.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Graphics (cs.GR); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2609.22361 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.191Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).