Tydra is a hybrid Transformer‑State Space Model that interleaves attention and SSM layers for tabular in‑context learning. It achieves a 30% reduction in inference time compared to the Transformer‑only TabPFN while preserving most of its predictive performance. On 30 OpenML datasets, Tydra also outperforms a Hydra model that is roughly ten times larger, demonstrating that hybrid architectures can balance accuracy and efficiency for tabular foundation models.
By Mieszko Komisarczyk, Saurabh Mathur, Maurice Kraus, Sriraam Natarajan, Kristian Kersting
Support-Compiled Feature Folding (SCFF) is a training‑free inference framework that addresses the feature‑side scaling dilemma in tabular foundation models by routing support‑ranked features through bounded leaves of the native encoder, checking residual evidence, and merging encoded messages for a single contextual prediction. This approach transforms quadratic pairwise mixing into linear‑in‑width work with a bounded local working set, achieving dataset‑macro accuracy and NLL improvements across six backbones on an 18‑dataset wide‑table slice. SCFF delivers significant GPU‑memory savings (median 2.09×–2.36×) and, when constrained by a peak‑memory ceiling, further boosts accuracy by up to 4.06 points over the widest single‑leaf baseline.
whyItMatters":"SCFF demonstrates that memory‑efficient inference can simultaneously improve accuracy and reduce resource usage in tabular foundation models, offering a practical solution for deploying these models at scale."
By Tian Zhou, Beverly Jin, Xue Wang, Linxiao Yang, Wenwei Wang, Bingqing Peng, Mengni Ye, Jinjie Gu, Liang Sun
TabICLv2 is a new state‑of‑the‑art tabular foundation model that outperforms existing methods on regression and classification tasks. It relies on a synthetic data generation engine for diverse pretraining, architectural innovations such as a scalable softmax attention, and optimized training protocols that replace AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 surpasses the current best model, RealTabPFN‑2.5, without any tuning, while also being faster and capable of handling million‑scale datasets with limited GPU memory.
By Jingang Qu, David Holzm\"uller, Ga\"el Varoquaux, Marine Le Morvan
arXiv:2608. 01400v1 Announce Type: new Abstract: Tabular foundation models, driven by in-context learning, have rapidly grown in quality and popularity.
By Rasa Hosseinzadeh, Alex Labach, Zexin Xue, Shuyi Han, Valentin Thomas, Anthony L. Caterini
arXiv:2607. 05380v1 Announce Type: new Abstract: In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures.
By Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, Artem Babenko
arXiv:2608. 10837v1 Announce Type: cross Abstract: The strong performance of foundation models for tabular tasks comes at substantial inference costs.
By Mykhailo Koshil, Matthias Feurer, Katharina Eggensperger
arXiv:2606. 07345v1 Announce Type: new Abstract: Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples.
By Si-Yang Liu, Han-Jia Ye
arXiv:2609.13202v1 Announce Type: new
Abstract: Feature engineering has long been a cornerstone of tabular machine learning. Tabular foundation models (TFMs) are pretrained on a wide range of tabular...
By Yifan WU, Pinjun Dong, Jiran Tao, Binyan Jiang
arXiv:2606. 02106v1 Announce Type: new Abstract: We present a single classification pipeline that combines an Equiangular Tight Frame (ETF) preprocessing stage with a tabular foundation model for in-context inference, applied identically across modalities once data is mapped to fixed vector representations.
By Julien Lafrance
arXiv:2605.12904v2 Announce Type: replace
Abstract: Tabular foundation models (TFMs) have emerged as a powerful paradigm for in-context learning on structured data, enabling direct prediction on new...
By Yilong Chen, Xueying Ding, Leman Akoglu
arXiv:2607. 27546v1 Announce Type: new Abstract: Tabular Foundation Models, such as TabPFN, have received a large amount of recent attention due to their performance on in-context tabular machine learning tasks, which often exceeds classical baselines.
By Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon, Anna Leontjeva, Simon Lucey
Xiaomi-TabLDM is a tabular foundation model that performs classification and regression via in-context learning without task‑specific fine‑tuning. It is pretrained solely on synthetic data from structural causal models, achieving top‑ranked regression results on multiple benchmarks while reducing training and prediction time compared to leading models. The architecture incorporates a three‑stage training strategy, dual‑stream feature grouping, lightweight attention residuals, and sparse mixture‑of‑experts, and it can further improve accuracy through test‑time compute scaling.
By TabLDM Team, Penghui Wang, Wei Liu, Hong Wang, Chengyue Huang, Yuxi Sun, Zirui Wang, Hongming Huang, Quan Wang, Chunxiao Liu, Erli Meng, Bin Wang