Metraipolis › archive › dc_49b873ffed9a

Planning to Learn — arXiv:2610.03667

Source
arXiv
Original
https://arxiv.org/abs/2610.03667
License (as archived)
CC0-1.0
Archived
2026-10-05T23:10:29.329Z
SHA-256 of the archived text
2000f8f78084239c085219fc61c7bfce669f300a7523dfd170ccec0b78cd2093
Stable URL of this page
https://metraipolis.com/documento/dc_49b873ffed9a

How to cite

Planning to Learn — arXiv:2610.03667 — arXiv, https://arxiv.org/abs/2610.03667 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_49b873ffed9a (Metraipolis archive dc_49b873ffed9a, sha256 2000f8f78084239c085219fc61c7bfce669f300a7523dfd170ccec0b78cd2093)
BibTeX
@misc{dc49b873ffed9a,
  title = {Planning to Learn — arXiv:2610.03667},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.03667},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_49b873ffed9a; sha256 2000f8f78084239c085219fc61c7bfce669f300a7523dfd170ccec0b78cd2093},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03667. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Policy-gradient methods are central to modern reinforcement learning, including LLM post-training. When they struggle, the usual suspects are exploration, credit assignment and action-sampling noise. Classification has none of them. A classifier is a policy whose expected reward, its \emph{expected accuracy}, is the probability it assigns to the correct label, and because that label is known, the policy gradient is exact and smooth. Yet exact policy gradient loses to cross-entropy, even on expected accuracy. The exact gradient is myopic: it values an update only by what it buys now, but each update also sets where the next one starts, so an update's value depends on how much learning remains. Viewed this way, cross-entropy is patient accuracy, the total error an example would pay if its log-odds rose at unit speed forever, while exact policy gradient is the zero-horizon limit. Truncating this total at the learning that remains yields the horizon loss, a one-line change that moves from cross-entropy toward exact policy gradient as training runs out. In a simple allocation model, it provably escapes the trap that catches each endpoint. On MNIST and on ImageNet with ResNet-50, ResNet-101 and ViT-S/16, the horizon loss improves top-1 accuracy over cross-entropy at a flat learning rate, and the gain grows with label noise.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2610.03667 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T23:01:38.720Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).