Metraipolis › archive › dc_26e6cfc798bb

Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models — arXiv:2610.40328

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.40328
License (as archived)
CC0-1.0
Archived
2026-10-31T06:17:26.093Z
SHA-256 of the archived text
1578f5ecec45a6a92f33ca58fa696514fd8087fe7f3af46545099f36956dc195
Stable URL of this page
https://metraipolis.com/documento/dc_26e6cfc798bb

How to cite

Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models — arXiv:2610.40328 — arXiv (en), https://arxiv.org/abs/2610.40328 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_26e6cfc798bb (Metraipolis archive dc_26e6cfc798bb, sha256 1578f5ecec45a6a92f33ca58fa696514fd8087fe7f3af46545099f36956dc195)
BibTeX
@misc{dc26e6cfc798bb,
  title = {Factorized Representations of Concurrent Temporal Variables in Pretrained Language Models — arXiv:2610.40328},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.40328},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_26e6cfc798bb; sha256 1578f5ecec45a6a92f33ca58fa696514fd8087fe7f3af46545099f36956dc195},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40328. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Language models often need to maintain multiple contextual variables that evolve independently within a document. Prior work has identified structured representations of individual temporal or contextual quantities, but their joint organization remains less understood. We study this question using controlled documents with independently phased language-switch and topic-change timers. Across ten pretrained checkpoints, both timers are linearly decodable from the same hidden states. Removing a timer-associated centroid plane substantially reduces access to its associated variable while having little effect on the concurrent timer, although the planes are nonorthogonal and contain cross-timer information. Separate timer contributions also explain most reproducible variation among joint-state centroids. These signatures persist from 3 to 51 timer values and across Qwen3 checkpoints from 0.6B to 14B. Schedule position affects continuation probabilities, but the tested centroid displacement produces no consistent timer-specific output effect. Together, these results support approximate factorization based on selective accessibility and approximately additive joint-state geometry in overlapping subspaces.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.40328 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:23.420Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).