Metraipolis › archive › dc_63d29f290f0d

VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning — arXiv:2611.05638

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.05638
License (as archived)
CC0-1.0
Archived
2026-11-05T18:17:28.081Z
SHA-256 of the archived text
d54ccdf18e03db462f5a1b0a9bca2f6c8e6eaca3610589e9887712eac9d8ac79
Stable URL of this page
https://metraipolis.com/documento/dc_63d29f290f0d

How to cite

VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning — arXiv:2611.05638 — arXiv (en), https://arxiv.org/abs/2611.05638 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_63d29f290f0d (Metraipolis archive dc_63d29f290f0d, sha256 d54ccdf18e03db462f5a1b0a9bca2f6c8e6eaca3610589e9887712eac9d8ac79)
BibTeX
@misc{dc63d29f290f0d,
  title = {VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning — arXiv:2611.05638},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05638},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_63d29f290f0d; sha256 d54ccdf18e03db462f5a1b0a9bca2f6c8e6eaca3610589e9887712eac9d8ac79},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05638. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speaker-recognition benchmarks test how accurately a model recovers the speaker label of a recording, treating voice identity as a property of the person who produced it. However, these benchmarks do not evaluate how humans perceive voice identity. Whether a listener hears two voices as the same person has consequences for intellectual property rights, cybersecurity, and everyday life. We introduce the Voice Identity Perception benchmark (VIPBench), which contains 124,876 same-or-different judgments from 1,290 listeners on 9,800 recording pairs from 100 public figures. The pairs include real recordings, voice clones generated by a text-to-speech system, and morphs between two speakers' voices. We also define four tasks that evaluate the alignment between speaker-representation models and human identity perception. The first two tasks measure how well a model's similarity score predicts the proportion of listeners who respond "same" and the majority response. The other two measure whether a model and listeners find the same speaker pairs similar, and whether a model's relationship to listener judgments holds equally for real and synthetic recordings. We evaluate eleven publicly available speech models. Listener judgments diverge from speaker labels most often for voice clones: in 41% of responses, listeners judged a clone and a real recording of the same speaker to be different people. No model reaches the noise ceiling; the best model explains 61% of the explainable variance. Models trained with speaker labels align with listeners better than self-supervised models do, yet a model with a 12% error rate on speaker labels aligns with listeners as well as models with error rates below 2%. VIPBench enables the evaluation of speaker models against human perception on real, cloned, and morphed speech, and our results motivate training speaker models directly on listener judgments.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Sound (cs.SD) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2611.05638 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-05T18:12:49.030Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).