Metraipolis › archive › dc_371c2734cfa9
Do Tabular Foundation Models Still Need Feature Engineering? — arXiv:2609.13202
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13202
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T00:10:18.572Z
- SHA-256 of the archived text
c9a069d0627bcb765087553bf64d5a8a07e93ce190436f854b9d5d35a4f777c2- Stable URL of this page
https://metraipolis.com/documento/dc_371c2734cfa9
How to cite
Do Tabular Foundation Models Still Need Feature Engineering? — arXiv:2609.13202 — arXiv (en), https://arxiv.org/abs/2609.13202 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_371c2734cfa9 (Metraipolis archive dc_371c2734cfa9, sha256 c9a069d0627bcb765087553bf64d5a8a07e93ce190436f854b9d5d35a4f777c2)
BibTeX
@misc{dc371c2734cfa9,
title = {Do Tabular Foundation Models Still Need Feature Engineering? — arXiv:2609.13202},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13202},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_371c2734cfa9; sha256 c9a069d0627bcb765087553bf64d5a8a07e93ce190436f854b9d5d35a4f777c2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13202. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Feature engineering has long been a cornerstone of tabular machine learning. Tabular foundation models (TFMs) are pretrained on a wide range of tabular datasets and applied via in-context learning. Their rise raises a natural question: does manual feature construction still matter as these models become more capable? To answer this, we perform a controlled study across several versions of two major TFM families, testing a wide range of existing feature engineering techniques on benchmark datasets from TabArena. We find a consistent pattern: feature engineering gains are concentrated in earlier model generations and become negligible for the strongest models. These results suggest that stronger TFMs depend less on explicitly engineered input representations. In a complementary experiment, however, adding in-context information from related datasets still improves performance. Our findings indicate a shift in the source of performance gains for stronger TFMs: re-representing existing inputs becomes less effective, while providing additional task-relevant context remains beneficial. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.13202 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T23:54:14.605Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).