Metraipolis › archive › dc_7ddf87af5e1a
Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation — arXiv:2609.07965
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.07965
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T12:48:31.443Z
- SHA-256 of the archived text
d096c0538a6b56bf4eae9861ca58909ca0e7eb99b2d06f735d8e57c5d218e75a- Stable URL of this page
https://metraipolis.com/documento/dc_7ddf87af5e1a
How to cite
Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation — arXiv:2609.07965 — arXiv (en), https://arxiv.org/abs/2609.07965 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7ddf87af5e1a (Metraipolis archive dc_7ddf87af5e1a, sha256 d096c0538a6b56bf4eae9861ca58909ca0e7eb99b2d06f735d8e57c5d218e75a)
BibTeX
@misc{dc7ddf87af5e1a,
title = {Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation — arXiv:2609.07965},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.07965},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7ddf87af5e1a; sha256 d096c0538a6b56bf4eae9861ca58909ca0e7eb99b2d06f735d8e57c5d218e75a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.07965. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sign Language Translation has advanced with deep learning, yet evaluations remain largely signer-dependent, with overlapping signers across train/dev/test. This raises concerns about whether models truly generalise or instead rely on signer-specific regularities. We conduct signer-fold cross-validation on GFSLT-VLP, GASLT, and SignCL, three leading, publicly available, gloss-free SLT models, on CSL-Daily and PHOENIX14T. Under signer-independent evaluation, performance drops sharply: on PHOENIX14T, GFSLT-VLP falls from BLEU-4 21.44 to 3.59 and ROUGE-L 42.49 to 11.89; GASLT from 15.74 to 8.26; and SignCL from 22.74 to 3.66. We also observe that in CSL-Daily many target sentences are performed by multiple signers, so common splits can place identical sentences in both training and test, inflating absolute scores by rewarding recall of recurring sentences rather than genuine generalisation. These findings indicate that signer-dependent evaluation can substantially overestimate SLT capability. We recommend: (1) adopting signer-independent protocols to ensure generalisation to unseen signers; (2) restructuring datasets to include explicit signer-independent, sentence-disjoint splits for consistent benchmarking; and (3) reporting both signer-dependent and signer-independent results together with train-test sentence overlap to improve transparency and comparability. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.07965 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T12:30:25.151Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).