Metraipolis › archive › dc_5ba314c19002

NepTTS-Bench: Native-Speaker Benchmarking and NepaliMOS for Nepali Text-to-Speech — arXiv:2611.03718

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.03718
License (as archived): CC0-1.0
Archived: 2026-11-04T12:28:53.250Z
SHA-256 of the archived text: 96b9ad19d93ab89ccbe06a9544dd74fb8db3e2901a58b06383c1ac7d4262916e
Stable URL of this page: https://metraipolis.com/documento/dc_5ba314c19002

How to cite

NepTTS-Bench: Native-Speaker Benchmarking and NepaliMOS for Nepali Text-to-Speech — arXiv:2611.03718 — arXiv (en), https://arxiv.org/abs/2611.03718 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5ba314c19002 (Metraipolis archive dc_5ba314c19002, sha256 96b9ad19d93ab89ccbe06a9544dd74fb8db3e2901a58b06383c1ac7d4262916e)

BibTeX

@misc{dc5ba314c19002,
  title = {NepTTS-Bench: Native-Speaker Benchmarking and NepaliMOS for Nepali Text-to-Speech — arXiv:2611.03718},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03718},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5ba314c19002; sha256 96b9ad19d93ab89ccbe06a9544dd74fb8db3e2901a58b06383c1ac7d4262916e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03718. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Nepali TTS has reached a point where multiple systems are available, but there is no shared evaluation to compare them. NepTTS-Bench was created to address what existing automated and manual evaluation systems lack for Nepali TTS. A controlled benchmark with phonological grounding which contains 205 sentences (77 minimal pairs targeting aspiration, retroflex/dental contrasts, nasalization, gemination, and schwa deletion) which is rated by 180 speakers. There are 6,962 ratings for 10 TTS configurations and two human references. Along with this 7 ASR systems were evaluated for round-trip CER. SCOREQ was also run as an automated baseline. The key finding is that automated evaluation like SCOREQ prefers proprietary systems like ElevenLabs but native listeners prefer Piper. This is contributed by the fact that most of the systems not properly tuned for Nepali tend to show Hindi phonological behaviour and our minimal pairs are designed to surface such contrasts. We also present NepaliMOS predictor which is fine tuned on collected ratings. NepaliMOS achieves system-level rho = 0.90 on the dense panel but drops to utterance-level rho = 0.43 under leave-one-system-out evaluation. This is statistically significant and confirms that NepaliMOS captures genuine signal but cross-provider transfer is limited. A human calibration is recommended for new systems for which we release a Python evaluation package for practitioners to run the benchmark against new systems.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.03718 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:26:47.211Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).