Metraipolis › archive › dc_f778763c5ada

Leveraged Learning: entropy cleared per bit received — arXiv:2609.08054

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.08054
License (as archived)
CC0-1.0
Archived
2026-10-16T15:02:02.595Z
SHA-256 of the archived text
c8384ff6b9ddecdd4f5619f83e6953eae1183ba24605fe35c01ffb0545f594e4
Stable URL of this page
https://metraipolis.com/documento/dc_f778763c5ada

How to cite

Leveraged Learning: entropy cleared per bit received — arXiv:2609.08054 — arXiv (en), https://arxiv.org/abs/2609.08054 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f778763c5ada (Metraipolis archive dc_f778763c5ada, sha256 c8384ff6b9ddecdd4f5619f83e6953eae1183ba24605fe35c01ffb0545f594e4)
BibTeX
@misc{dcf778763c5ada,
  title = {Leveraged Learning: entropy cleared per bit received — arXiv:2609.08054},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08054},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f778763c5ada; sha256 c8384ff6b9ddecdd4f5619f83e6953eae1183ba24605fe35c01ffb0545f594e4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08054. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An expository essay containing some new results. A learner holds a prior belief over Boolean maps that answer a finite set of $Q$ questions, and receives answers one by one. Each answer carries surprisal and can clear predictive uncertainty about both the question asked and questions still unasked. We quantify this effect by the leverage: table entropy cleared per bit of surprisal received. Taking expectations over the truth prior and the question order, define the aggregate leverage as the ratio of expected uncertainty cleared to expected surprisal received. It equals one for independent answers and can exceed one for correlated answers. At finite size, this ratio is determined exactly by a single sequence: the mean entropy $G_\ell$ of the answers to $\ell$ questions. As the number of input bits grows at fixed asked fraction $t=\ell/Q$, a limiting increment profile $\gamma(t)$ determines the macroscopic learning curve. With $\eta_0$ the limiting initial entropy per question, the leverage becomes $L(t)= \frac{\eta_0-(1-t)\gamma(t)} {\int_0^t\gamma(x),dx}$. For exchangeable priors, de Finetti's representation gives a constant bulk profile $\gamma(t)$: deduction is confined to a boundary layer at $t=0$, and the leverage is forced to a hyperbolic form, as surprisal grows linearly. By contrast, we construct a simplicity prior with nontrivial bulk learning by grading Boolean maps by their polynomial degree over $\mathbb{F}_2$ and allocating weight across degree classes through a CDF $F$. Reed-Muller capacity then yields $\gamma(t)=1-F(t)$. This realizes any nonincreasing profile taking values in $[0,1]$, together with the corresponding macroscopic leverage curve.

Tema degli esploratori: matematica e informatica teorica (retro-catalogo arXiv: annunci del 2026-09-09 in math.IT).
Categorie arXiv: Statistical Mechanics (cond-mat.stat-mech) ; Information Theory (cs.IT).
Pagina del lavoro: https://arxiv.org/abs/2609.08054 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T14:48:50.745Z dalla pagina https://arxiv.org/catchup/math.IT/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).