Metraipolis › archive › dc_2bdf700bc19f

ZEMA: confidence-calibrated signed descent with only one state — arXiv:2611.04987

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.04987
License (as archived): CC0-1.0
Archived: 2026-11-04T18:54:17.863Z
SHA-256 of the archived text: 6a8236839175dee2e733c3bbb34141d325019717df385971bd84860108ae7eed
Stable URL of this page: https://metraipolis.com/documento/dc_2bdf700bc19f

How to cite

ZEMA: confidence-calibrated signed descent with only one state — arXiv:2611.04987 — arXiv (en), https://arxiv.org/abs/2611.04987 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2bdf700bc19f (Metraipolis archive dc_2bdf700bc19f, sha256 6a8236839175dee2e733c3bbb34141d325019717df385971bd84860108ae7eed)

BibTeX

@misc{dc2bdf700bc19f,
  title = {ZEMA: confidence-calibrated signed descent with only one state — arXiv:2611.04987},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.04987},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2bdf700bc19f; sha256 6a8236839175dee2e733c3bbb34141d325019717df385971bd84860108ae7eed},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04987. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent work characterizes the Adam optimizer as sign descent modulated by gradient uncertainty: the update is the sign of the batch gradient, scaled by how strongly the batch supports that sign. We apply this view directly and introduce ZEMA (Z-score Evidence Momentum Adaptation), which measures that support with a per-coordinate $z$-score computed from the per-example gradients in the current batch, converts it into a sign probability, and smooths the resulting update with a single exponential moving average. Because the probability is formed before smoothing rather than after, ZEMA keeps one state per parameter instead of Adam's two, at a $4.6$ increase in per-step time; the per-example variance it requires can also be estimated from a small fraction of the batch. In fine-tuning and pretraining experiments, ZEMA is competitive with Adam, reduces the variability of the validation loss, and in some settings recovers the sign of the full-dataset gradient more consistently. We close with two directions this opens: extending confidence adaptation beyond the sign geometry, to methods such as Muon, and to differentially private training.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.04987 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T18:35:37.468Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).