Metraipolis › archive › dc_2500edb01693

Forget who you Forgot: Speaker Unlearning to Prevent Re-Identification in Zero-Shot Text-to-Speech — arXiv:2609.27399

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.27399
License (as archived)
CC0-1.0
Archived
2026-10-08T09:12:35.933Z
SHA-256 of the archived text
95c3442155d8800f585bf43e586f91696d358ceed8eb063099fd87e356e45ca2
Stable URL of this page
https://metraipolis.com/documento/dc_2500edb01693

How to cite

Forget who you Forgot: Speaker Unlearning to Prevent Re-Identification in Zero-Shot Text-to-Speech — arXiv:2609.27399 — arXiv (en), https://arxiv.org/abs/2609.27399 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2500edb01693 (Metraipolis archive dc_2500edb01693, sha256 95c3442155d8800f585bf43e586f91696d358ceed8eb063099fd87e356e45ca2)
BibTeX
@misc{dc2500edb01693,
  title = {Forget who you Forgot: Speaker Unlearning to Prevent Re-Identification in Zero-Shot Text-to-Speech — arXiv:2609.27399},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27399},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2500edb01693; sha256 95c3442155d8800f585bf43e586f91696d358ceed8eb063099fd87e356e45ca2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27399. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent zero-shot text-to-speech (ZS-TTS) systems can reproduce a speaker's voice with high fidelity from only a few seconds of reference speech, raising concerns over unauthorized voice cloning and impersonation. Speaker identity unlearning has recently emerged as an approach to selectively suppress this capability for speakers who opt out while preserving synthesis capability for other speakers. Although existing approaches reduce speaker similarity, preventing re-identification often faces severe degradation of speech quality. Motivated by this observation, we propose GUARD, a lightweight speaker identity unlearning framework that combines a learned speaker gate with speaker-agnostic activation steering on a frozen TTS backbone. The steering vectors are optimized using group-relative reward optimization to shift outputs from forget speakers toward population-level impostor similarity while preserving intelligibility and speech naturalness. On CosyVoice2, GUARD reduces forget-speaker similarity from 0.541 to 0.103 and re-identification accuracy in a 150-speaker gallery from 73.5% to 0.5%, while preserving retain-speaker reproduction. The results demonstrate that similarity reduction alone may not fully characterize successful speaker identity unlearning and highlight re-identification as a complementary criterion for its evaluation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI).
Categorie arXiv: Sound (cs.SD) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.27399 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:13:26.674Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).