Metraipolis › archive › dc_84240d52bd17
English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck — arXiv:2609.17554
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.17554
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T13:49:11.521Z
- SHA-256 of the archived text
6f39a452fa540b99ecb1cf5623dcf2af18431d537505ab801e7b819cf8ed7964- Stable URL of this page
https://metraipolis.com/documento/dc_84240d52bd17
How to cite
English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck — arXiv:2609.17554 — arXiv (en), https://arxiv.org/abs/2609.17554 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_84240d52bd17 (Metraipolis archive dc_84240d52bd17, sha256 6f39a452fa540b99ecb1cf5623dcf2af18431d537505ab801e7b819cf8ed7964)
BibTeX
@misc{dc84240d52bd17,
title = {English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck — arXiv:2609.17554},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.17554},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_84240d52bd17; sha256 6f39a452fa540b99ecb1cf5623dcf2af18431d537505ab801e7b819cf8ed7964},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17554. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In English all-words word sense disambiguation (WSD), the labels, not the models, have become the bottleneck: frontier LLMs are accurate enough that the errors surviving in the gold standard decide benchmark rankings -- in the test sets we score on and, as we show causally, in the corpus we train on. We release lexEN, a WSD evaluation benchmark built as a conservative, human-adjudicated correction layer over Maru2022's ALL_NEW benchmark (211 labels changed, 56 removed), and SenseBench, an auditable LLM WSD evaluation harness and living leaderboard (57 models, 192 runs). The task is inventory-constrained multiple choice (the model picks from the supplied WordNet senses), so the reported accuracies are a ceiling on what models achieve without that help. On lexEN-v1 the frontier LLMs converge near 95% (best, 95.6%), the top three families are statistically indistinguishable, and accuracy trades off against reasoning effort and cost across a ~2,500x price span. Relabeling SemCor with frontier models and retraining BEM, ESCHER, and ConSeC unchanged lifts them by several F1 points on test sets the relabeling never touched; we release the relabeled corpora and Glite LENS, a 298M bi-encoder trained on the repaired labels -- to our knowledge the strongest reported (83.6 Raganato ALL, 87.4 Maru ALL_NEW) -- serving at ~$0.13 per million items. On hard items, fine-grained WordNet senses are partly ill-posed even for experts (three-reviewer Fleiss kappa=0.537); coarsening raises annotator agreement and model accuracy together across four inventories, placing a top model inside the expert agreement band at coarse granularity (statistically equivalent under three of four) but significantly below it at fine. The binding constraint is now cost. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.17554 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T13:48:24.268Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).