Metraipolis › archive › dc_fb504eee0c8e

Sparsity-driven emergence of functional modularity in multitask neural networks — arXiv:2610.32153

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32153
License (as archived)
CC0-1.0
Archived
2026-10-26T19:51:44.446Z
SHA-256 of the archived text
4c83e82d862205b6b4b7c1916f6333f43bb973a56afec3629fe1f00d0f69a8ab
Stable URL of this page
https://metraipolis.com/documento/dc_fb504eee0c8e

How to cite

Sparsity-driven emergence of functional modularity in multitask neural networks — arXiv:2610.32153 — arXiv (en), https://arxiv.org/abs/2610.32153 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fb504eee0c8e (Metraipolis archive dc_fb504eee0c8e, sha256 4c83e82d862205b6b4b7c1916f6333f43bb973a56afec3629fe1f00d0f69a8ab)
BibTeX
@misc{dcfb504eee0c8e,
  title = {Sparsity-driven emergence of functional modularity in multitask neural networks — arXiv:2610.32153},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32153},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fb504eee0c8e; sha256 4c83e82d862205b6b4b7c1916f6333f43bb973a56afec3629fe1f00d0f69a8ab},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32153. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Neural networks can learn multiple tasks, but how learning organizes shared computation into specialized and reusable components remains poorly understood. In this work, we show that sparsity inducing regularization promotes functional modularity in multitask networks without predefined module boundaries. Using functional profiling in multilayer perceptrons and a lightweight transformer, we identified compact, task-selective subnetworks, termed functional tokens. Across rule-based task families, selectively retaining these tokens preserves their associated functions while reducing performance on other tasks. Such specialization develops preferentially in deeper layers during training, while earlier layers retain shared computation. Regularization strength and form shape module compactness and specialization, while task weighting alters functional allocation. Moreover, extracted functional tokens can be recombined through alternative pathways to reconstruct a learned composite computation without retraining, preserving their connectivity and learned weights. Our results suggest that sparsity-constrained optimization shapes not only parameter use but also the organization and reuse of learned computation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.32153 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:45:33.392Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).