Metraipolis › archive › dc_ce5a6d249e29
Using LMs to Model the Effects of Context and Coreference during Sentence Comprehension — arXiv:2609.32119
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32119
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:49:57.700Z
- SHA-256 of the archived text
6124b2ac6bec6195b64e82f9bbcde1f59859cc730baaef7af1be0b6b7f325d00- Stable URL of this page
https://metraipolis.com/documento/dc_ce5a6d249e29
How to cite
Using LMs to Model the Effects of Context and Coreference during Sentence Comprehension — arXiv:2609.32119 — arXiv (en), https://arxiv.org/abs/2609.32119 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce5a6d249e29 (Metraipolis archive dc_ce5a6d249e29, sha256 6124b2ac6bec6195b64e82f9bbcde1f59859cc730baaef7af1be0b6b7f325d00)
BibTeX
@misc{dcce5a6d249e29,
title = {Using LMs to Model the Effects of Context and Coreference during Sentence Comprehension — arXiv:2609.32119},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32119},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce5a6d249e29; sha256 6124b2ac6bec6195b64e82f9bbcde1f59859cc730baaef7af1be0b6b7f325d00},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32119. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Language models (LMs) are often used as a tool to model human language processing. Recent studies suggest that severely restricting LMs' context window improves their fit to human psycholinguistic data by simulating human working memory constraints. However, it is possible that this strict memory-decay approach overlooks humans' reliance on long-range structural representations, such as discourse structre. In this work, we systematically vary the context window size of GPT-2 across four large-scale naturalistic English reading-time datasets and observe a U-shaped relationship: Although restricted contexts (< 20 tokens) successfully capture local memory limitations, expanded contexts (500--1,000 tokens) ultimately yield the highest overall psycholinguistic fit. To investigate the mechanism driving this benefit, we conduct a counterfactual inference-time experiment that disrupts cross-sentential entity chains by pronominalizing repeated discourse entities. Obscuring these structural linkages significantly degrades the predictive power of larger context windows by 20% to 40%. Our experiments demonstrate that tracking long-range coreference relations is one important factor for the alignment between LM surprisal and human reading behavior, and approximate the extent to which human comprehenders use global discourse relations during language processing. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.32119 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T21:27:08.391Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).