Metraipolis › archive › dc_93ab4fc32f98

Neural Dynamics as the Composition of Quantized Units — arXiv:2609.32487

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32487
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:38.479Z
SHA-256 of the archived text
17b55ff16185fc756daf469f39c108218965943c14b7ca04105b961a5840f5d2
Stable URL of this page
https://metraipolis.com/documento/dc_93ab4fc32f98

How to cite

Neural Dynamics as the Composition of Quantized Units — arXiv:2609.32487 — arXiv (en), https://arxiv.org/abs/2609.32487 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_93ab4fc32f98 (Metraipolis archive dc_93ab4fc32f98, sha256 17b55ff16185fc756daf469f39c108218965943c14b7ca04105b961a5840f5d2)
BibTeX
@misc{dc93ab4fc32f98,
  title = {Neural Dynamics as the Composition of Quantized Units — arXiv:2609.32487},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32487},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_93ab4fc32f98; sha256 17b55ff16185fc756daf469f39c108218965943c14b7ca04105b961a5840f5d2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32487. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Deep learning is commonly interpreted at two levels: the macroscopic, through aggregate trends in loss summarized by scaling laws, and the microscopic, through neurons, features, and circuits. A central challenge is understanding how these levels connect, so that we can explain how elementary computations compose and collectively shape macroscopic behavior. To this end, we study an intermediate abstraction in which training is described as the ordered acquisition of quanta: reusable computations acquired suddenly and binary-activated across examples to reduce loss. By approximating population-gradient updates, we derive quanta's acquisition dynamics. This yields an acquisition priority governed by demand, how frequently a computation is required across examples, and conditional complexity, how difficult that computation is to acquire given those already available. In a Boolean compositional task, we derive predictions for acquisition order and show how staggered discrete acquisitions can produce smooth aggregate loss and, under certain geometries of quanta composition, give rise to scaling laws. We then train a Transformer to map numerals to English number names and recover candidate quanta from its checkpoint trajectory. From these units, we construct a model that preserves much of the Transformer's behavior while exposing interpretable latent computations and acquisition dynamics consistent with the theory. Separately, the quanta structure can serve as training targets to improve transformer generalization. Together, these results suggest the quanta abstraction can provide useful computational atoms for studying a variety of macroscopic phenomena.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32487 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.701Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).