Metraipolis › archive › dc_9e40ff43c2f4
Double Descent in Gradient Boosting Decision Trees via Split-Candidate Scaling — arXiv:2608.03111
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.03111
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T12:42:21.607Z
- SHA-256 of the archived text
3de090f9d2ba52c86ae36e735cb888ef8b4878f6fdc14492ee1546d2946abd98- Stable URL of this page
https://metraipolis.com/documento/dc_9e40ff43c2f4
How to cite
Double Descent in Gradient Boosting Decision Trees via Split-Candidate Scaling — arXiv:2608.03111 — arXiv (en), https://arxiv.org/abs/2608.03111 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9e40ff43c2f4 (Metraipolis archive dc_9e40ff43c2f4, sha256 3de090f9d2ba52c86ae36e735cb888ef8b4878f6fdc14492ee1546d2946abd98)
BibTeX
@misc{dc9e40ff43c2f4,
title = {Double Descent in Gradient Boosting Decision Trees via Split-Candidate Scaling — arXiv:2608.03111},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.03111},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9e40ff43c2f4; sha256 3de090f9d2ba52c86ae36e735cb888ef8b4878f6fdc14492ee1546d2946abd98},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03111. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Double descent is commonly studied by scaling an explicit capacity parameter, such as neural-network width. For gradient boosting decision trees (GBDTs), however, an analogous single-axis capacity parameter has not been established. We propose the number of split candidates as an operational capacity parameter for GBDTs. Holding other training controls fixed, increasing the split-candidate budget refines the feature-quantization grid and expands the dictionary of root-to-leaf paths from which boosting selects its updates. To analyze this expansion, we construct an empirical tree-kernel diagnostic that summarizes how candidate-induced paths group the training examples. A regime in which the empirical kernel rank grows toward the sample size and very small positive eigenvalues emerge exposes noise-sensitive directions; in this regime, test error peaks before decreasing again at larger split-candidate budgets. This perspective predicts that deeper trees should reach the regime with fewer split candidates, larger training sets should require finer grids, and label noise should make the peak more pronounced. Experiments support these predictions and show test-error peaks at intermediate split-candidate budgets across XGBoost, LightGBM, and CatBoost, whereas a random-forest control improves monotonically under the same split-candidate sweep. Taken together, our analysis and experiments support split-candidate scaling as a single-axis capacity intervention for studying GBDTs and suggest that the observed double descent arises from an interaction between candidate-induced geometry and boosting dynamics. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.03111 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T12:31:30.340Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).