Metraipolis › archive › dc_fc53c8a8c7ca

Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing — arXiv:2608.06424

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.06424
License (as archived)
CC0-1.0
Archived
2026-10-19T09:47:07.179Z
SHA-256 of the archived text
e9fd83becac40f20e8d2195c4da377d60abe21fa817b9bf7c61303d9ad9dd910
Stable URL of this page
https://metraipolis.com/documento/dc_fc53c8a8c7ca

How to cite

Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing — arXiv:2608.06424 — arXiv (en), https://arxiv.org/abs/2608.06424 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc53c8a8c7ca (Metraipolis archive dc_fc53c8a8c7ca, sha256 e9fd83becac40f20e8d2195c4da377d60abe21fa817b9bf7c61303d9ad9dd910)
BibTeX
@misc{dcfc53c8a8c7ca,
  title = {Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing — arXiv:2608.06424},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.06424},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc53c8a8c7ca; sha256 e9fd83becac40f20e8d2195c4da377d60abe21fa817b9bf7c61303d9ad9dd910},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.06424. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speech recordings often contain missing, corrupted, or incorrect regions that must be reconstructed or modified without re-synthesizing the entire utterance. Speech inpainting restores missing segments, whereas speech editing replaces spoken content according to an edited transcript. Both tasks require the generated speech to express the intended words while remaining consistent with the surrounding speaker identity, prosody, timing, and recording conditions. Discrete diffusion is particularly well suited to these tasks because it can iteratively refine masked tokens while jointly conditioning on both left and right acoustic context. We introduce SIEDD, a discrete diffusion framework for text-guided speech inpainting and editing over hierarchical codec tokens. Its core architecture, HiCoDD, follows the RVQ generation order by representing previously generated codebooks as clean, committed acoustic context and applying diffusion only to the current refinement codebook. This separation enables leakage-free joint training while matching sequential coarse-to-fine inference. The model further combines phoneme-level conditioning, span-localized classifier-free guidance, and duration prediction to support both fixed-duration inpainting and variable-duration text edits. On the RealEdit benchmark, SIEDD achieves the best overall speech-editing performance among the evaluated methods. It also outperforms the evaluated autoregressive baselines across all speech-inpainting settings, on both single and multiple gaps. These results demonstrate that explicitly modeling the codec hierarchy substantially improves context-preserving speech reconstruction and editing. See our full code at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-10 in cs.CL).
Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.06424 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:46:37.254Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).