Metraipolis › archive › dc_fdc342c93ea8
Thinking in Depth: Retrospective Inference for Tabular Foundation Models — arXiv:2610.10317
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.10317
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T12:36:19.331Z
- SHA-256 of the archived text
3f699d15bf12c750c2068cabbd79ce32cd9d1799a22742f48191a7794df0ba32- Stable URL of this page
https://metraipolis.com/documento/dc_fdc342c93ea8
How to cite
Thinking in Depth: Retrospective Inference for Tabular Foundation Models — arXiv:2610.10317 — arXiv (en), https://arxiv.org/abs/2610.10317 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fdc342c93ea8 (Metraipolis archive dc_fdc342c93ea8, sha256 3f699d15bf12c750c2068cabbd79ce32cd9d1799a22742f48191a7794df0ba32)
BibTeX
@misc{dcfdc342c93ea8,
title = {Thinking in Depth: Retrospective Inference for Tabular Foundation Models — arXiv:2610.10317},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.10317},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fdc342c93ea8; sha256 3f699d15bf12c750c2068cabbd79ce32cd9d1799a22742f48191a7794df0ba32},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10317. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tabular foundation models (TFMs) are pretrained across diverse tabular tasks and make predictions on a new table at inference time using its labeled examples as context. Most recent TFMs perform such in-context prediction with stacked Transformer layers, repeatedly transforming how examples are represented and compared. By tracing individual queries through several strong TFMs, we find that predictive refinement is highly uneven across depth and is often concentrated in later layers. This uneven refinement motivates us to reconsider how intermediate representations are constructed and reused throughout the network. We introduce Retro, a tabular foundation model based on retrospective inference, where later stages can explicitly revisit and recombine intermediate information produced earlier in the network. Retro organizes this process around two complementary operations: which intermediate information to revisit, and how the resulting contextual update should be shaped for each query. Attention Residuals address the former by adaptively reweighting contributions from different depths, while query-conditioned Gated Attention addresses the latter by modulating the attention output element-wise across representation dimensions. Our analysis shows that Retro shifts predictive refinement earlier and more broadly across depth, with different stages revising different subsets of queries in a pattern suggestive of multi-view refinement. Across TabArena, TALENT, and RelArena, Retro ranks among the top three and lies on the Pareto frontier. These results indicate that directly reusing intermediate representations provides a practical way to better exploit depth in TFMs. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.10317 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T12:14:15.331Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).