Metraipolis › archive › dc_fe74038909fa

Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent — arXiv:2609.13040

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.13040
License (as archived)
CC0-1.0
Archived
2026-10-16T03:15:55.355Z
SHA-256 of the archived text
033d7df66e3c07c1b9aa1c13a7123a57207ffa010539c2537399eebb6156c9ad
Stable URL of this page
https://metraipolis.com/documento/dc_fe74038909fa

How to cite

Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent — arXiv:2609.13040 — arXiv (en), https://arxiv.org/abs/2609.13040 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fe74038909fa (Metraipolis archive dc_fe74038909fa, sha256 033d7df66e3c07c1b9aa1c13a7123a57207ffa010539c2537399eebb6156c9ad)
BibTeX
@misc{dcfe74038909fa,
  title = {Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent — arXiv:2609.13040},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.13040},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fe74038909fa; sha256 033d7df66e3c07c1b9aa1c13a7123a57207ffa010539c2537399eebb6156c9ad},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13040. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study loss-based filtering for finite-sum optimization with a subset of corrupted component functions whose gradients may be highly unreliable. Motivated by minimum-loss-based SGD (min-$k$-loss) and quantile-based methods for corrupted linear systems, we propose and analyze a general loss-filtering framework -- Quantile-\(k\)-Loss SGD (Q\(k\)L-SGD) -- that samples \(k\) component losses at each iteration and updates using an index chosen uniformly from the lower empirical \(q\)-quantile. We prove linear convergence of this family of methods under standard convexity assumptions, requiring the sample size to scale with the number of corruptions and a subset strong-convexity threshold. For the cases when large enough sampling is impossible or undesirable, we give a complementary small-sample probabilistic analysis that covers any sample size $k$ and the convergence behavior depends on the probability of selecting an outlier and on the curvature of the selected good step. Experiments on polynomial regression, regularized logistic regression, and regularized hinge loss show that intermediate quantiles often outperform both standard SGD and min-\(k\)-loss SGD. In particular, min-\(k\) often stalls by repeatedly selecting nearly solved components, while intermediate quantiles retain robustness and produce more informative updates.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Numerical Analysis (math.NA).
Pagina del lavoro: https://arxiv.org/abs/2609.13040 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T03:05:44.081Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).