The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.
By Junwei Ma, Nour Shaheen, Alex Labach, Amine Mhedhbi, Frank Hutter, Anthony L. Caterini, Valentin Thomas
arXiv:2511. 15941v2 Announce Type: replace-cross Abstract: Tabular data underpins decisions across science, industry, and public services.
By David Bonet, Mar\c{c}al Comajoan Cara, Alvaro Calafell, Daniel Mas Montserrat, Alexander G. Ioannidis
arXiv:2403. 10318v3 Announce Type: replace Abstract: Recent advances have shifted the paradigm of tabular learning toward tabular foundation models, yet their accuracy relies on a heavy inference cost that scales poorly with context size.
By Naili Xing, Shaofeng Cai, Lingze Zeng, Jiaqi Zhu, Peng Lu, Jian Pei, Beng Chin Ooi
arXiv:2603. 10823v2 Announce Type: replace-cross Abstract: Deep generative models can help with data scarcity and privacy by producing synthetic training data, but they struggle in low-data, imbalanced tabular settings to fully learn the complex data distribution.
By Xiaofeng Lin, Seungbae Kim, Zhuoya Li, Zachary DeSoto, Charles Fleming, Guang Cheng
arXiv:2605. 24417v2 Announce Type: replace Abstract: Supervised classification on tabular data remains a central machine learning task, but its dependence on large labeled datasets limits its applicability in data-scarce settings.
By Daria Grushina, Kseniia Kuvshinova, Alina Kostromina, Aziz Temirkhanov, Mile Mitrovic, Dmitry Simakov
The paper introduces a pre‑training pipeline that creates transformer‑based imputation specialists for tabular data with specific missingness patterns. By featurizing entries, generating synthetic data with configurable missingness modules, and fitting on millions of synthetic tables, the pipeline produces pattern‑specific models that outperform dedicated methods for each missingness pattern. A default model trained only on MCAR data, TabImpute, remains robust across all tested patterns, and the authors release the pipeline, models, and a new benchmark of 42 datasets and 11 missingness patterns.
By Jacob Feitelberg, Dwaipayan Saha, Kyuseong Choi, Zaid Ahmad, Anish Agarwal, Raaz Dwivedi
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table, rather than large synthetic or real datasets. It finds that a table’s usefulness for downstream tasks is mainly determined by the number of features, not instances, and that fine‑grained column‑level preprocessing improves performance while dataset‑level filtering does not. The authors propose a task‑centric, retrieval‑based view of in‑context generalization, suggesting that effective TFMs identify and aggregate relevant examples from the provided context.
arXiv:2604. 05635v2 Announce Type: replace Abstract: Numerical preprocessing remains a critical component of tabular deep learning, as the representation of continuous features can strongly affect downstream performance.
By Manish Kumar, Anton Frederik Thielmann, Christoph Weisser, Benjamin S\"afken
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.
By Nour Shaheen, Junwei Ma, Alex Labach, Frank Hutter, Valentin Thomas, Anthony L. Caterini
arXiv:2606. 30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.
By Lennart Purucker, Andrej Tschalzev, Nick Erickson, Gioia Blayer, David Holzm\"uller, Alan Arazi, Alexander Pfefferle, Mustafa Tajjar, Ga\"el Varoquaux, Frank Hutter
arXiv:2602.11139v2 Announce Type: replace
Abstract: Tabular foundation models, such as TabPFNv2 and TabICL, have recently dethroned gradient-boosted trees at the top of predictive benchmarks, demonst...
By Jingang Qu, David Holzm\"uller, Ga\"el Varoquaux, Marine Le Morvan
TabNSM is a scalable regression framework for large-scale, high-dimensional tabular data that builds on sparse-attention and mixer architectures. Its core component, the Adaptive Sparse Interaction Module (ASIM), combines foreground feature discovery, sparse local interaction encoding, and Feature-Token Mixing to achieve near-linear complexity. For regression, TabNSM adds a Multi-Stage Regression Head, GridLoss (an ordinal-aware soft-binning objective), and RISE (a difficulty-aware sampling strategy), achieving strong predictive performance and practical scalability across nine real-world benchmarks, especially on high-dimensional and heterogeneous datasets.
By Ali Eslamian, Qiang Cheng