The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.
By Nour Shaheen, Junwei Ma, Alex Labach, Frank Hutter, Valentin Thomas, Anthony L. Caterini
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table, rather than large synthetic or real datasets. It finds that a table’s usefulness for downstream tasks is mainly determined by the number of features, not instances, and that fine‑grained column‑level preprocessing improves performance while dataset‑level filtering does not. The authors propose a task‑centric, retrieval‑based view of in‑context generalization, suggesting that effective TFMs identify and aggregate relevant examples from the provided context.
The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.
By Junwei Ma, Nour Shaheen, Alex Labach, Amine Mhedhbi, Frank Hutter, Anthony L. Caterini, Valentin Thomas
arXiv:2510. 04567v3 Announce Type: replace-cross Abstract: Graph Neural Networks (GNNs) are powerful tools for processing relational data but often struggle to generalize to unseen graphs, giving rise to the development of Graph Foundational Models (GFMs).
By Weishuo Ma, Yanbo Wang, Xiyuan Wang, Lei Zou, Muhan Zhang
arXiv:2609.05955v1 Announce Type: new
Abstract: Tabular foundation models have become powerful graph learners. Systems such as G2T-FM and GraphPFN encode each node as a feature row and make predictio...
By Mingqi Yang, Zidong Guo, Jihui Yang, Wenming Zuo
arXiv:2609.36417v1 Announce Type: new
Abstract: Relational in-context learning (ICL) uses labeled support examples and their linked relational context to predict labels for new queries. This creates...
By Roshan Reddy Upendra, Alexandre Dorais, Joe Meyer, Andrew Pouret, Anastasios Lambrianos Stappas, Dinesh Katupputhur Ramprasath, Viswanath Ganapathy, Tom Palczewski, Minghua Li
arXiv:2602. 13697v2 Announce Type: replace-cross Abstract: Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes.
By Linjie Xu, Yanlin Zhang, Quan Gan, Minjie Wang, David Wipf
arXiv:2607. 07500v1 Announce Type: cross Abstract: Time series classification (TSC) is dominated by a two-stage paradigm: train a feature encoder -- either from scratch on the target dataset or via pretraining on large corpora -- and then fit a task-specific classifier on top.
By Jaris K\"uken, Shi Bin Hoo, Martin Mr\'az, Frank Hutter, Lennart Purucker
arXiv:2605. 31272v2 Announce Type: replace Abstract: As predictive models are increasingly deployed in high-stakes settings such as credit approval, there is a growing need for post-hoc methods that provide recourse to affected individuals.
By Wenshuo Dong, Jiaming Zhang, Shaopeng Fu, Hongbin Lin, Di Wang, Lijie Hu
arXiv:2606. 03040v1 Announce Type: new Abstract: Relational databases underpin modern enterprise, scientific, and healthcare systems, yet predictive machine learning on such data remains challenging due to their multi-table, heterogeneous, and temporal structure.
By Phillip Jiang
arXiv:2605.12904v2 Announce Type: replace
Abstract: Tabular foundation models (TFMs) have emerged as a powerful paradigm for in-context learning on structured data, enabling direct prediction on new...
By Yilong Chen, Xueying Ding, Leman Akoglu
arXiv:2607. 29129v1 Announce Type: new Abstract: Relational Foundation Models (RFMs) require large-scale synthetic relational databases for pretraining, but existing approaches tightly couple data generation with the model training pipeline.
By Mohammad Sadeq Abolhasani, Viswanath Ganapathy