Metraipolis › archive › dc_40b35f74019f

Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting — arXiv:2609.11131

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11131
License (as archived)
CC0-1.0
Archived
2026-10-16T05:44:03.899Z
SHA-256 of the archived text
073cc4109ab86c76a4891045f5dfd243e7b37838db5f711685b4f7e611225b95
Stable URL of this page
https://metraipolis.com/documento/dc_40b35f74019f

How to cite

Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting — arXiv:2609.11131 — arXiv (en), https://arxiv.org/abs/2609.11131 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40b35f74019f (Metraipolis archive dc_40b35f74019f, sha256 073cc4109ab86c76a4891045f5dfd243e7b37838db5f711685b4f7e611225b95)
BibTeX
@misc{dc40b35f74019f,
  title = {Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting — arXiv:2609.11131},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11131},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40b35f74019f; sha256 073cc4109ab86c76a4891045f5dfd243e7b37838db5f711685b4f7e611225b95},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11131. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Human simultaneous interpreting (SI) is commonly assessed with analytic rubrics separating meaning transfer, delivery quality, and temporal synchrony, yet no automatic metric is designed for rubric-aligned segment-level SI evaluation. We construct a professionally annotated corpus of 1,101 SI segments with scores for meaning transfer (LQ), delivery quality (EXP), and perceived latency (LAT). We show that structured LLM prompting and scalar supervision collapse rubric dimensions, yielding near-zero correlation with human ratings and strong cross-dimension coupling. To isolate supervision structure under identical backbone capacity, we introduce dual regression heads on a LoRA-adapted COMET-KIWI encoder. On a held-out talk-level test set, the model achieves Pearson correlations of 0.388 (LQ) and 0.301 (EXP), improving over frozen COMET-KIWI. Given low absolute rater agreement, we interpret results relative to human consistency and target stable ranking signals for formative assessment.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.11131 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:23:57.001Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).