Metraipolis › archive › dc_5031589f69cd

VoiceCodeBench: Evaluating Exact Structured-Token Recovery in Automatic Speech Recognition — arXiv:2608.28916

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.28916
License (as archived)
CC0-1.0
Archived
2026-10-17T08:06:48.523Z
SHA-256 of the archived text
468728983af5c607594c6ae09f4e41cd8611ac965e6b5efb43e8749bb59d4bce
Stable URL of this page
https://metraipolis.com/documento/dc_5031589f69cd

How to cite

VoiceCodeBench: Evaluating Exact Structured-Token Recovery in Automatic Speech Recognition — arXiv:2608.28916 — arXiv (en), https://arxiv.org/abs/2608.28916 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5031589f69cd (Metraipolis archive dc_5031589f69cd, sha256 468728983af5c607594c6ae09f4e41cd8611ac965e6b5efb43e8749bb59d4bce)
BibTeX
@misc{dc5031589f69cd,
  title = {VoiceCodeBench: Evaluating Exact Structured-Token Recovery in Automatic Speech Recognition — arXiv:2608.28916},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.28916},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5031589f69cd; sha256 468728983af5c607594c6ae09f4e41cd8611ac965e6b5efb43e8749bb59d4bce},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28916. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automatic speech recognition is usually evaluated with word error rate (WER), although voice workflows often require exact written values. VoiceCodeBench measures whether transcripts preserve identifiers, paths, commands, and other structured tokens needed by downstream software. It contains 300 human-recorded English workplace segments (5.59 hours, 85 speakers) and 1,482 audited entities across 26 types and eight domains. Under a raw-audio-only protocol, we evaluate 19 batch and streaming systems using WER, Canonical Token/Entity Match (CTEM), and strict segment-level Task Success Rate (TSR). Across systems, WER has little rank agreement with CTEM (Spearman $\rho=-0.28$) or TSR ($\rho=-0.22$). The best CTEM and TSR are 91.8% and 68.7%. Even the strongest system therefore leaves nearly one-third of recordings with an unrecovered critical value. Symbol-, separator-, and boundary-sensitive entities account for most errors.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.28916 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:47:52.661Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).