Metraipolis › archive › dc_fc8153afba2d

Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation — arXiv:2610.07786

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.07786
License (as archived): CC0-1.0
Archived: 2026-10-07T11:41:22.730Z
SHA-256 of the archived text: 6dbb24b8f538c2688ce84e41f0df7d89b21b84264254d23fc6bb678e78f4d472
Stable URL of this page: https://metraipolis.com/documento/dc_fc8153afba2d

How to cite

Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation — arXiv:2610.07786 — arXiv (en), https://arxiv.org/abs/2610.07786 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc8153afba2d (Metraipolis archive dc_fc8153afba2d, sha256 6dbb24b8f538c2688ce84e41f0df7d89b21b84264254d23fc6bb678e78f4d472)

BibTeX

@misc{dcfc8153afba2d,
  title = {Extending Pathwise Gradients to Discrete Random Variables via Finite-Order Relaxation — arXiv:2610.07786},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.07786},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc8153afba2d; sha256 6dbb24b8f538c2688ce84e41f0df7d89b21b84264254d23fc6bb678e78f4d472},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07786. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Pathwise gradients are preferred for continuous random variables because they are unbiased, low variance, and work with a single sample. For discrete variables, however, the pathwise identity cannot generally be exact for every differentiable function. We propose a general framework to construct finite-order exact pathwise gradient estimators for a range of common discrete variables such as Poisson. The estimator is the least-norm solution among all solutions that are unbiased for polynomials of degree at most. The resulting estimators preserve the hard forward sample, require no temperature tuning, and can be implemented in a few lines of codes. Against other admissible solutions, our estimator is unique and minimizes weight variance; in contrast, prior works use categorical variables or augmented representations to approximate non-categorical variables that induces excess variance and computations. To understand approximation bias for functions beyond the prescribed class, we also derive a non-asymptotic bias bound. In experiments our low order methods match or improve tuned baselines across linear, nonlinear and hierarchical latent-variable models, while out-speeding competitors in every runtime benchmark.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2610.07786 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:19:24.784Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).