The paper investigates how synthetic pretraining priors used in tabular foundation models (TFMs) influence downstream performance. By reconstructing the synthetic data generators of four TFMs and comparing their generated tasks to two popular tabular benchmarks using structural descriptors, the authors measure structural coverage and normalized density. They find that some generators provide broader and denser support for benchmark tasks, and that stronger synthetic-to-benchmark support generally correlates with better model performance.
arXiv:2605.18971v2 Announce Type: replace-cross Abstract: What determines the quality of a tabular foundation model? Unlike language or vision, tabular foundation models acquire their inductive biase...
arXiv:2605.09424v2 Announce Type: replace Abstract: Generative modelling is a demanding test of foundation models, because it requires robust, holistic representation learning for a given data modali...
arXiv:2610.16990v1 Announce Type: cross Abstract: Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. T...
arXiv:2606. 29241v1 Announce Type: new Abstract: Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining.
The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.
TabMF is a framework that repurposes pretrained tabular foundation models (TFMs) as encoders for supervised datasets, adding a dataset-level readout and fine‑tuning via contrastive post‑training on synthetic views. The authors instantiate TabMF on TabICLv2 and TabPFN‑2.5 and evaluate the resulting representations on six tasks involving unseen real datasets. The approach achieves the best aggregate performance among both learned and classical dataset representations, excelling in dataset retrieval, compact benchmark construction, and distribution‑shift severity assessment.
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.
The paper investigates how test‑time compute techniques—adaptation, aggregation, and context construction—can improve predictions of pretrained tabular foundation models (TFMs). It introduces DiagScale, a lightweight adaptation method that trains a minuscule fraction of parameters yet matches full fine‑tuning gains, and shows that selective aggregation of predictions from diverse preprocessing variants can reduce error, while broader context expansion offers inconsistent benefits. The study concludes that adaptation and selective aggregation consistently yield benchmark‑level improvements, but their combined use demands more computation, guiding strategy selection based on available resources.
arXiv:2606. 30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table, rather than large synthetic or real datasets. It finds that a table’s usefulness for downstream tasks is mainly determined by the number of features, not instances, and that fine‑grained column‑level preprocessing improves performance while dataset‑level filtering does not. The authors propose a task‑centric, retrieval‑based view of in‑context generalization, suggesting that effective TFMs identify and aggregate relevant examples from the provided context.
arXiv:2406. 08311v3 Announce Type: replace-cross Abstract: Existing evaluations of tabular synthesis models rely primarily on low-order statistics and downstream task performance, leaving multivariate causal relationships that go beyond pairwise correlations largely unmeasured.
Xiaomi-TabLDM is a tabular foundation model that performs classification and regression via in-context learning without task‑specific fine‑tuning. It is pretrained solely on synthetic data from structural causal models, achieving top‑ranked regression results on multiple benchmarks while reducing training and prediction time compared to leading models. The architecture incorporates a three‑stage training strategy, dual‑stream feature grouping, lightweight attention residuals, and sparse mixture‑of‑experts, and it can further improve accuracy through test‑time compute scaling.