arXiv Machine Learning

NOMADD: Numerical Optimization of Models Adapting to Data Drift

arXiv:2608. 02845v1 Announce Type: new Abstract: Tabular model performance degrades when feature distributions change over time or the relationship between features and outcome variables change over time, known as data drift and concept drift, respectively.

arXiv Machine Learning
Sep 17

TabICLv2: A better, faster, scalable, and open tabular foundation model

TabICLv2 is a new state‑of‑the‑art tabular foundation model that outperforms existing methods on regression and classification tasks. It relies on a synthetic data generation engine for diverse pretraining, architectural innovations such as a scalable softmax attention, and optimized training protocols that replace AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 surpasses the current best model, RealTabPFN‑2.5, without any tuning, while also being faster and capable of handling million‑scale datasets with limited GPU memory.

By Jingang Qu, David Holzm\"uller, Ga\"el Varoquaux, Marine Le Morvan
arXiv Machine Learning
Sep 17

TabPFN-3.5: Technical Report

The report introduces TabPFN‑3.5, a new flagship tabular foundation model that outperforms its predecessor TabPFN‑3 and all existing baselines on a wide range of tabular tasks. It achieves state‑of‑the‑art performance on standard tabular prediction in TabArena and extends to practical scenarios such as non‑i.i.d. data, temporal or grouped splits, tables containing strings, text, images, high‑cardinality categorical features, and wide tables. Variants like TabPFN‑3.5‑Fast, TabPFN‑3.5‑Plus, and TabPFN‑3.5‑Thinking offer faster inference, expanded multimodal capabilities, and further speed improvements up to 12× faster than the previous Thinking mode.

By Benjamin J\"ager, Nick Erickson, L\'eo Grinsztajn, Felix Birkel, Klemens Fl\"oge, Oscar Key, K\"ur\c{s}at Kaya, Jonas K\"ubler, Ad\`ele Frankel, Tobias Schr\"oder, Anurag Garg, Jan Hendrik Metzen, David Salinas, Simon Bing, Kristina Collins, Tuana \c{C}elik, Vahid Balazadeh, Lydia Sidhoum, Tom\'as Pereda, Brendan Roof, Andrej Tschalzev, Siyuan Guo, Philipp Singer, Lennart Purucker, Jake Robertson, Marie Salmon, Philipp Jund, Jerry Chen, Diana Kriuchkova, Arthur Cahu, Eliott Kalfon, Adrian Hayler, Georg Grab, Vitor Monteiro, Lilly Wehrhahn, Dominik Safaric, Clara Cornu, Alan Arazi, Rylee Grace, Simone Alessi, Mihir Manium, Bernhard Sch\"olkopf, Yann LeCun, Madelon Hulsebos, Sauraj Gambhir, Noah Hollmann, Frank Hutter
arXiv AI
Jun 30

Beyond IID: How General Are Tabular Foundation Models, Really?

arXiv:2606. 30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.

By Lennart Purucker, Andrej Tschalzev, Nick Erickson, Gioia Blayer, David Holzm\"uller, Alan Arazi, Alexander Pfefferle, Mustafa Tajjar, Ga\"el Varoquaux, Frank Hutter