Metraipolis › archive › dc_3bfc00746cba
Modular TTT: Rethinking Test-Time Training as Composable Modules — arXiv:2608.07110
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.07110
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:47:11.588Z
- SHA-256 of the archived text
f40f902246cbe7940807b5ac19c8fc19c9df1207e6b4f5dce08bddd799074f14- Stable URL of this page
https://metraipolis.com/documento/dc_3bfc00746cba
How to cite
Modular TTT: Rethinking Test-Time Training as Composable Modules — arXiv:2608.07110 — arXiv (en), https://arxiv.org/abs/2608.07110 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3bfc00746cba (Metraipolis archive dc_3bfc00746cba, sha256 f40f902246cbe7940807b5ac19c8fc19c9df1207e6b4f5dce08bddd799074f14)
BibTeX
@misc{dc3bfc00746cba,
title = {Modular TTT: Rethinking Test-Time Training as Composable Modules — arXiv:2608.07110},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.07110},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3bfc00746cba; sha256 f40f902246cbe7940807b5ac19c8fc19c9df1207e6b4f5dce08bddd799074f14},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07110. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Test-time training (TTT) views sequence modeling as an online learning problem in which fast weights are updated by an internal learning rule. Despite the growing number of TTT variants, existing approaches typically hard-code each variant separately, which makes it difficult to design new TTT methods and to isolate the role of each component. To address this, we propose Modular TTT, a framework that represents the inner learner as a directed acyclic graph and exposes the fast-weight network, loss function, learning rate, weight decay, and normalization as explicit design dimensions. Modular TTT automatically composes primitive-level train-view forward, train-view backward, and causal query-view rules into the full graph-level TTT computation, including the fast-weight state transition. Using Modular TTT, we systematically ablate the components of TTT and find that small learning-rate initialization, weight decay, and a single-layer nonlinearity improve performance, while MSE and inner-product losses perform similarly. Deeper fast-weight networks and normalization tend to hurt performance because they induce excessively large activations, while residual connections and gating provide little measurable benefit. Guided by these findings, we train the best resulting variant as 410M- and 1.45B-parameter models on 100B tokens, and observe training loss and benchmark performance comparable to Gated DeltaNet. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-10 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2608.07110 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:46:37.269Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).