Metraipolis › archive › dc_fd82d5332eb8

HyperMC: Multi-Fidelity Hyperparameter Tuning for Stochastic Gradient MCMC — arXiv:2609.02138

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02138
License (as archived)
CC0-1.0
Archived
2026-10-17T05:01:18.735Z
SHA-256 of the archived text
32262b4714c413ea046c8cac7c7c1cb746511874243f373fb695e5760aff14d3
Stable URL of this page
https://metraipolis.com/documento/dc_fd82d5332eb8

How to cite

HyperMC: Multi-Fidelity Hyperparameter Tuning for Stochastic Gradient MCMC — arXiv:2609.02138 — arXiv (en), https://arxiv.org/abs/2609.02138 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fd82d5332eb8 (Metraipolis archive dc_fd82d5332eb8, sha256 32262b4714c413ea046c8cac7c7c1cb746511874243f373fb695e5760aff14d3)
BibTeX
@misc{dcfd82d5332eb8,
  title = {HyperMC: Multi-Fidelity Hyperparameter Tuning for Stochastic Gradient MCMC — arXiv:2609.02138},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02138},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fd82d5332eb8; sha256 32262b4714c413ea046c8cac7c7c1cb746511874243f373fb695e5760aff14d3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02138. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Stochastic gradient Markov chain Monte Carlo (SGMCMC) methods enable scalable Bayesian inference, but their performance depends strongly on hyperparameters such as the step size, mini-batch size, and number of leapfrog steps. Since most SGMCMC algorithms lack a Metropolis-Hastings acceptance rate, standard acceptance-based tuning methods are not directly applicable. We propose HyperMC, a multi-fidelity tuning framework that combines Hyperband-style resource allocation with kernel Stein discrepancy (KSD) evaluation. By running multiple successive-halving brackets, HyperMC balances broad exploration of a continuous hyperparameter space with increasingly accurate evaluation of promising configurations under a fixed computational budget. We further introduce Robust HyperMC, which uses global grid initialization followed by elite-guided local refinement to reduce sensitivity to random candidate generation and noisy finite-budget evaluations. Under suitable approximation and concentration conditions for the estimated KSD, we establish that the successive-halving component selects a near-optimal configuration among the sampled candidates with high probability and derive a sufficient computational budget for successful selection. Experiments on logistic regression, probabilistic matrix factorization, and Bayesian neural networks show that HyperMC improves posterior approximation or predictive calibration relative to MAMBA, grid search, and heuristic baselines, while Robust HyperMC yields more stable and reproducible tuning results.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.LG).
Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG); Computation (stat.CO).
Pagina del lavoro: https://arxiv.org/abs/2609.02138 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T04:09:43.364Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).