Metraipolis › archive › dc_a5da607dbd70
Gloss-Free Representation Learning for Cross-Dataset Sign Spotting — arXiv:2608.11332
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.11332
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T05:27:20.163Z
- SHA-256 of the archived text
e247e659c977d08e329c7ec1a88bb3fb2dc47388265db3e8cabcc7e0e66456d7- Stable URL of this page
https://metraipolis.com/documento/dc_a5da607dbd70
How to cite
Gloss-Free Representation Learning for Cross-Dataset Sign Spotting — arXiv:2608.11332 — arXiv (en), https://arxiv.org/abs/2608.11332 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a5da607dbd70 (Metraipolis archive dc_a5da607dbd70, sha256 e247e659c977d08e329c7ec1a88bb3fb2dc47388265db3e8cabcc7e0e66456d7)
BibTeX
@misc{dca5da607dbd70,
title = {Gloss-Free Representation Learning for Cross-Dataset Sign Spotting — arXiv:2608.11332},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.11332},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a5da607dbd70; sha256 e247e659c977d08e329c7ec1a88bb3fb2dc47388265db3e8cabcc7e0e66456d7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11332. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Sign-language research for resource-constrained languages is often limited by the cost of dense linguistic labels such as glosses, temporal boundaries, and sign order. Broadcast news offers a practical alternative by pairing continuous signing with spoken-language transcripts, but this supervision is weak since text and signing are loosely aligned. Morphologically rich languages such as Turkish add further difficulty, as the same lexical meaning can appear in many inflected forms while some derived forms should remain distinct. We study whether weak transcript-based supervision can pretrain a reusable sign encoder in this setting, where poor text normalization can fragment pseudo-gloss targets and weaken representation learning. Unlike prior pseudo-gloss pipelines designed mainly to improve translation, we test whether the pretrained encoder transfers as a reusable representation for cross-dataset sign spotting. We pretrain on TSL-News, a new Turkish broadcast corpus, using pseudo-gloss labels derived from transcripts rather than manual annotation, comparing rule-based morphological lemmatization with constrained LLM-assisted normalization over a fixed vocabulary. We evaluate the learned representations via cross-dataset sign spotting on a new TSL Spotting Benchmark built from the TSL Dictionary corpus. The LLM-assisted encoder raises top-5 temporal localization mean IoU from 0.235 to 0.465, with 56.2% of examples reaching an IoU of at least 0.50; a frequency analysis suggests this gain is not mainly driven by memorizing frequent pseudo-gloss labels. In a downstream translation check, the same pretraining improves BLEU-4 from 9.60 to 11.04 and ROUGE from 23.48 to 27.43. These results show that loosely aligned broadcast data can provide effective weak supervision for learning sign representations that capture both lexical content and temporal structure. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2608.11332 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T05:15:39.188Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).