Metraipolis › archive › dc_e04a449d6762

Chronologic: Measuring Language Models' Ability to Represent the Past — arXiv:2609.23178

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.23178
License (as archived)
CC0-1.0
Archived
2026-10-09T06:49:45.055Z
SHA-256 of the archived text
6ec5e58c5469549a755646877efb3155e885f2006c7ecd2820999c1d8d33875f
Stable URL of this page
https://metraipolis.com/documento/dc_e04a449d6762

How to cite

Chronologic: Measuring Language Models' Ability to Represent the Past — arXiv:2609.23178 — arXiv (en), https://arxiv.org/abs/2609.23178 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e04a449d6762 (Metraipolis archive dc_e04a449d6762, sha256 6ec5e58c5469549a755646877efb3155e885f2006c7ecd2820999c1d8d33875f)
BibTeX
@misc{dce04a449d6762,
  title = {Chronologic: Measuring Language Models' Ability to Represent the Past — arXiv:2609.23178},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.23178},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e04a449d6762; sha256 6ec5e58c5469549a755646877efb3155e885f2006c7ecd2820999c1d8d33875f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23178. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Language models are appealing tools for research on the past. But to trust the evidence a model provides, researchers need to know whether its responses fit the period represented. Validation is challenging, because this is not a task living people ordinarily perform, and because many questions have multiple correct answers. We use historical texts to develop a benchmark for a model's representation of English-language contexts 1831-1930, relying on pairwise comparisons to multiple ground truths and strong distractors to score the hardest questions in an appropriately graduated way. We find that generative tasks are harder than discriminative ones; in fact, reasoning models can typically discern the weakness of their own generated answers. While models pretrained exclusively on historical text lead the pack when evaluated by answer likelihood, they cannot compete with commercial models in free generation. None of the models we tested represent historical contexts in a fully persuasive way yet, but progress toward that goal is evident.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.23178 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:44.977Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).