arXiv AI
Shaping the Prior: How Synthetic Task Distributions Determine Tabular Foundation Model Quality
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 10
From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models
The paper investigates how synthetic pretraining priors used in tabular foundation models (TFMs) influence downstream performance. By reconstructing the synthetic data generators of four TFMs and comparing their generated tasks to two popular tabular benchmarks using structural descriptors, the authors measure structural coverage and normalized density. They find that some generators provide broader and denser support for benchmark tasks, and that stronger synthetic-to-benchmark support generally correlates with better model performance.
arXiv AI
Jun 9
When Tabular Foundation Models Meet Strategic Tabular Data: A Prior Alignment Approach
arXiv:2605. 19662v2 Announce Type: replace Abstract: Tabular foundation models based on pretrained prior-data fitted networks~(PFNs) have shown strong generalization on diverse tabular tasks, but they are typically designed for \emph{non-strategic} settings where data distributions are independent of deployed classifiers.
arXiv Machine Learning
Oct 15
When Does Depth Help? Understanding Layerwise Inference in Tabular Foundation Models
arXiv:2610.19814v1 Announce Type: new Abstract: Transformer-based tabular foundation models (TFMs) achieve state-of-the-art performance on small to medium-sized tabular prediction tasks while process...
arXiv Machine Learning
Jun 30
Towards Evaluating Data Priors for Tabular Foundation Models
arXiv:2606. 29241v1 Announce Type: new Abstract: Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining.
arXiv Machine Learning
Oct 9
Test-Time Compute for Tabular Foundation Models: Mechanisms, Gains, and Limits
The paper investigates how test‑time compute techniques—adaptation, aggregation, and context construction—can improve predictions of pretrained tabular foundation models (TFMs). It introduces DiagScale, a lightweight adaptation method that trains a minuscule fraction of parameters yet matches full fine‑tuning gains, and shows that selective aggregation of predictions from diverse preprocessing variants can reduce error, while broader context expansion offers inconsistent benefits. The study concludes that adaptation and selective aggregation consistently yield benchmark‑level improvements, but their combined use demands more computation, guiding strategy selection based on available resources.
arXiv Machine Learning
Oct 15
TabFORGE: A Tabular Foundation Model for Generative Modelling
arXiv:2605.09424v2 Announce Type: replace Abstract: Generative modelling is a demanding test of foundation models, because it requires robust, holistic representation learning for a given data modali...