Metraipolis › archive › dc_26e6cfc798bb
Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models — arXiv:2610.40328
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.40328
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T06:17:26.093Z
- SHA-256 of the archived text
1578f5ecec45a6a92f33ca58fa696514fd8087fe7f3af46545099f36956dc195- Stable URL of this page
https://metraipolis.com/documento/dc_26e6cfc798bb
How to cite
Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models — arXiv:2610.40328 — arXiv (en), https://arxiv.org/abs/2610.40328 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_26e6cfc798bb (Metraipolis archive dc_26e6cfc798bb, sha256 1578f5ecec45a6a92f33ca58fa696514fd8087fe7f3af46545099f36956dc195)
BibTeX
@misc{dc26e6cfc798bb,
title = {Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models — arXiv:2610.40328},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.40328},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_26e6cfc798bb; sha256 1578f5ecec45a6a92f33ca58fa696514fd8087fe7f3af46545099f36956dc195},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40328. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Language models often need to maintain multiple contextual variables that evolve independently within a document. Prior work has identified structured representations of individual temporal or contextual quantities, but their joint organization remains less understood. We study this question using controlled documents with independently phased language-switch and topic-change timers. Across ten pretrained checkpoints, both timers are linearly decodable from the same hidden states. Removing a timer-associated centroid plane substantially reduces access to its associated variable while having little effect on the concurrent timer, although the planes are nonorthogonal and contain cross-timer information. Separate timer contributions also explain most reproducible variation among joint-state centroids. These signatures persist from 3 to 51 timer values and across Qwen3 checkpoints from 0.6B to 14B. Schedule position affects continuation probabilities, but the tested centroid displacement produces no consistent timer-specific output effect. Together, these results support approximate factorization based on selective accessibility and approximately additive joint-state geometry in overlapping subspaces. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.40328 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T06:07:23.420Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).