Metraipolis › archive › dc_da672ed9c8e7
TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models — arXiv:2609.01277
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.01277
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T06:34:03.552Z
- SHA-256 of the archived text
c3ce39bfad13c8e5c3bcd3bbd3378e5b2e9d941fe3132fe07d15f5af24c64113- Stable URL of this page
https://metraipolis.com/documento/dc_da672ed9c8e7
How to cite
TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models — arXiv:2609.01277 — arXiv (en), https://arxiv.org/abs/2609.01277 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da672ed9c8e7 (Metraipolis archive dc_da672ed9c8e7, sha256 c3ce39bfad13c8e5c3bcd3bbd3378e5b2e9d941fe3132fe07d15f5af24c64113)
BibTeX
@misc{dcda672ed9c8e7,
title = {TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models — arXiv:2609.01277},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.01277},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da672ed9c8e7; sha256 c3ce39bfad13c8e5c3bcd3bbd3378e5b2e9d941fe3132fe07d15f5af24c64113},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01277. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Although pretrained joint audio-visual diffusion models offer rich control over \emph{what} to generate, they provide no explicit control over \emph{when} an utterance should occur. To address this, we study \emph{inference-time speech scheduling}, a novel task that places coupled speech and visual articulation within user-specified begin--end intervals without finetuning the backbone model. We uncover two intrinsic properties of the denoising process that enable this task. First, a timing-sensitive text-to-audio cross-attention head exposes each utterance's model-implied source span along the latent timeline. Second, the predicted clean latent already organizes coupled speech and visual articulation, allowing their temporal placement to be edited without regenerating the content. Building on these discoveries, we propose \textbf{TimeSteer}, a training-free framework that localizes each utterance's source span through \textbf{Source Span Localization} and transfers the associated audio-visual latent content from the source interval to the specified target interval through \textbf{Region-Aware Latent Remapping}. We further introduce \textbf{SpeechShift}, the first benchmark for interval-level speech scheduling in joint audio-visual generation. Experiments across two representative backbones show that TimeSteer substantially improves interval controllability over training-free baselines while maintaining competitive overall generation quality.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2609.01277 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:15:12.880Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).