Metraipolis › archive › dc_0aa92b23c2ef

Multiclass Classification with Rare Useful Features: Fundamental Limits and the Optimality of Diversity Pursuit Higher Criticism Thresholding — arXiv:2611.02255

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02255
License (as archived)
CC0-1.0
Archived
2026-11-03T18:40:41.995Z
SHA-256 of the archived text
72f2cc5a515c55f873b1d3a90896f7227eb7eb056e14fe4e21f315ebd9ae69a3
Stable URL of this page
https://metraipolis.com/documento/dc_0aa92b23c2ef

How to cite

Multiclass Classification with Rare Useful Features: Fundamental Limits and the Optimality of Diversity Pursuit Higher Criticism Thresholding — arXiv:2611.02255 — arXiv (en), https://arxiv.org/abs/2611.02255 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0aa92b23c2ef (Metraipolis archive dc_0aa92b23c2ef, sha256 72f2cc5a515c55f873b1d3a90896f7227eb7eb056e14fe4e21f315ebd9ae69a3)
BibTeX
@misc{dc0aa92b23c2ef,
  title = {Multiclass Classification with Rare Useful Features: Fundamental Limits and the Optimality of Diversity Pursuit Higher Criticism Thresholding — arXiv:2611.02255},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02255},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0aa92b23c2ef; sha256 72f2cc5a515c55f873b1d3a90896f7227eb7eb056e14fe4e21f315ebd9ae69a3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02255. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We consider classification in high-dimensional feature spaces where only a vanishing fraction of features are useful and the number of classes may be large. While this rare-and-weak useful-feature regime is well understood in the binary setting, its multiclass version raises new questions: a feature can be useful for class discrimination without separating any specific pair of classes. We focus on a rare feature diversity model, in which most features are useless, but a small fraction promote discrimination among classes in a way that can be detected by a per-feature diversity test, such as one-way ANOVA or the Kruskal-Wallis test. For this setting, we propose Diversity Pursuit Higher Criticism Thresholding (DP-HCT), a feature-screening procedure that applies a higher-criticism threshold to the collection of p-values obtained from per-feature diversity tests. We justify DP-HCT under a Gaussian multiclass Rare-and-Weak Useful Feature model with class-mean contrasts of fixed magnitude in random directions, extending the binary model of Donoho and Jin (2008). When the number of classes is fixed, our model recovers the binary phase transition. When the number of classes grows logarithmically with the number of features, however, we derive a new and strictly harder phase transition, driven by the growing degrees of freedom and their effect on the useful-feature tail rate. We show that whenever classification beyond chance is possible, DP-HCT with ANOVA, followed by a simple classifier based on similarity to class means, achieves asymptotically perfect accuracy. Experiments on real multiclass classification benchmarks demonstrate substantial improvements, and numerical simulations confirm that the derived phase-transition curve accurately predicts the finite-sample behavior of DP-HCT in the growing-class regime.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG); Statistics Theory (math.ST).
Pagina del lavoro: https://arxiv.org/abs/2611.02255 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:54.762Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).