Metraipolis › archive › dc_031c540357ea

Binary Sparse Codes: Where are we on Energy Efficient Representations? — arXiv:2610.41933

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41933
License (as archived): CC0-1.0
Archived: 2026-11-03T05:41:01.186Z
SHA-256 of the archived text: bc4c1a8cbbcf9a0551c99f649666f0991ed7956d5d7e1e5e274d09bfad57ae82
Stable URL of this page: https://metraipolis.com/documento/dc_031c540357ea

How to cite

Binary Sparse Codes: Where are we on Energy Efficient Representations? — arXiv:2610.41933 — arXiv (en), https://arxiv.org/abs/2610.41933 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_031c540357ea (Metraipolis archive dc_031c540357ea, sha256 bc4c1a8cbbcf9a0551c99f649666f0991ed7956d5d7e1e5e274d09bfad57ae82)

BibTeX

@misc{dc031c540357ea,
  title = {Binary Sparse Codes: Where are we on Energy Efficient Representations? — arXiv:2610.41933},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41933},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_031c540357ea; sha256 bc4c1a8cbbcf9a0551c99f649666f0991ed7956d5d7e1e5e274d09bfad57ae82},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41933. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Biological neural systems achieve impressive computational capabilities under tight metabolic constraints, relying on sparse cortical representations that minimise total neural activity. This combination of efficiency and representational power has motivated growing interest in sparse coding as a computational strategy for artificial systems. Artificial models designed to produce binary sparse codes span three broad paradigms, theoretical methods grounded in efficient coding, deep learning methods trained via backpropagation, and biologically plausible methods that learn through local Hebbian rules, each yielding codes whose quality depends not only on achieving low activation density but also on preserving the informational structure of the input. The latter property, however, lacks an agreed-upon quantitative definition, leaving the field without a principled basis for comparing methods. We propose a set of metrics that formalise information preservation as similarity preservation, measuring how faithfully a binary sparse code reproduces the relational geometry of the input space, and apply this framework to compare four representative methods under matched conditions. The results reveal a consistent trade-off in which models that achieve high information preservation do so at the cost of sparsity, while models approaching the uniformity of an optimal random sparse code tend to sacrifice it. Competitive Sparse Codes, a biologically plausible approach, is the only architecture to occupy a region of near-maximal sparsity while retaining meaningful information preservation. These findings frame the simultaneous satisfaction of both factors as an open challenge for the field and provide a reusable evaluation framework for measuring progress toward it.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.41933 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T05:26:25.724Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).