← Back to all news
arXiv AI By Junwei Ma, Nour Shaheen, Alex Labach, Amine Mhedhbi, Frank Hutter, Anthony L. Caterini, Valentin Thomas

Generalization Can Emerge in Tabular Foundation Models From a Single Table

Read the original on arXiv AI →

The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

Hugging Face Trending Papers
Aug 18

Understanding the Surprising Generalization Properties of Tabular Foundation Models

The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table, rather than large synthetic or real datasets. It finds that a table’s usefulness for downstream tasks is mainly determined by the number of features, not instances, and that fine‑grained column‑level preprocessing improves performance while dataset‑level filtering does not. The authors propose a task‑centric, retrieval‑based view of in‑context generalization, suggesting that effective TFMs identify and aggregate relevant examples from the provided context.

llms
More like this →
arXiv Machine Learning
Aug 19

Understanding the Surprising Generalization Properties of Tabular Foundation Models

The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.

By Nour Shaheen, Junwei Ma, Alex Labach, Frank Hutter, Valentin Thomas, Anthony L. Caterini
llms
More like this →
arXiv Machine Learning
Oct 12

TabMF: Learning Dataset Representations with Tabular Foundation Models

TabMF is a framework that repurposes pretrained tabular foundation models (TFMs) as encoders for supervised datasets, adding a dataset-level readout and fine‑tuning via contrastive post‑training on synthetic views. The authors instantiate TabMF on TabICLv2 and TabPFN‑2.5 and evaluate the resulting representations on six tasks involving unseen real datasets. The approach achieves the best aggregate performance among both learned and classical dataset representations, excelling in dataset retrieval, compact benchmark construction, and distribution‑shift severity assessment.

By Rafiq Kamel, Nicolas Atienza, Aziz Fouch\'e, Nicola Cancedda, Josif Grabocka
benchmarks
More like this →
arXiv Machine Learning
Sep 30

TabFM: A Zero-Shot Foundation Model for Tabular Data

arXiv:2609.37959v1 Announce Type: new Abstract: Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We p...

By Weihao Kong, Erez Louidor Ilan, Shuxin Nie, Taman Narayan, Rajat Sen, Yichen Zhou, Deqing Fu, Samet Oymak, Abhimanyu Das
llmsbenchmarks
More like this →