Metraipolis › archive › dc_2cc468397440

Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models — arXiv:2609.27378

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.27378
License (as archived)
CC0-1.0
Archived
2026-10-08T09:12:33.385Z
SHA-256 of the archived text
22c262fef11b7d2424eabd22f273b5bb1595fef4c405bd87e62eeda400872262
Stable URL of this page
https://metraipolis.com/documento/dc_2cc468397440

How to cite

Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models — arXiv:2609.27378 — arXiv (en), https://arxiv.org/abs/2609.27378 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2cc468397440 (Metraipolis archive dc_2cc468397440, sha256 22c262fef11b7d2424eabd22f273b5bb1595fef4c405bd87e62eeda400872262)
BibTeX
@misc{dc2cc468397440,
  title = {Psychoacoustically Aligned Latent Smoothing for Adversarial Robustness of Full-Duplex Speech-to-Speech Dialogue Models — arXiv:2609.27378},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27378},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2cc468397440; sha256 22c262fef11b7d2424eabd22f273b5bb1595fef4c405bd87e62eeda400872262},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27378. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

End-to-end speech-to-speech dialogue models listen and speak simultaneously, so a continuously open acoustic channel is exposed to adversarial manipulation. We formalize imperceptible attacks on full-duplex agents as optimization over additive perturbations confined beneath the psychoacoustic masking threshold of the carrier speech, under three goals: targeted semantic hijacking, response suppression, and policy jailbreaking. Against an undefended Moshi-style agent, white-box attacks succeed in up to 91.7% of trials. We then introduce psychoacoustically aligned latent smoothing (PALS), which injects anisotropic Gaussian noise shaped by local codebook covariance at the residual-vector-quantized latent interface, with input noise shaped by the masking threshold constraining the attacker and trained by a Kullback--Leibler consistency objective. Deployed with no inference-time cost, PALS reduces hijack to 8.3%, mute to 11.2%, and jailbreak to 9.1% at clean quality within 2.3%. A Monte Carlo-smoothed variant certifies an ellipsoidal latent radius up to 0.616, a guaranteed floor that the empirical robustness far exceeds.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI).
Categorie arXiv: Sound (cs.SD) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2609.27378 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:13:26.666Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).