ONNX-Net introduces a universal representation for neural architectures using natural language descriptions, enabling instant performance prediction across diverse search spaces. The authors present ONNX-Bench, a benchmark of over 600k architecture–accuracy pairs compiled from open‑source NAS‑bench networks in ONNX format. Experiments demonstrate strong zero‑shot predictive performance with minimal pretraining, overcoming the limitations of cell‑based, graph‑encoded approaches.
By Shiwen Qin, Alexander Auras, Shay B. Cohen, Elliot J. Crowley, Michael Moeller, Linus Ericsson, Jovita Lukasik
The paper introduces In-Table Prediction (ITB), a self‑supervised task where deep neural networks learn to predict any column in a table from the remaining columns. It proposes a novel neural layer to handle missing continuous values, generates synthetic datasets with controlled column relationships, and evaluates three architectures—MLP, ResNet, and Transformer—showing that attention‑based Transformers perform best when ample training data and large embeddings are used. The study is limited to synthetic, small‑column tables and is presented as an initial investigation rather than a comprehensive real‑world analysis.
By Xiao Zhao, Daniela Oelke
arXiv:2511. 15941v2 Announce Type: replace-cross Abstract: Tabular data underpins decisions across science, industry, and public services.
By David Bonet, Mar\c{c}al Comajoan Cara, Alvaro Calafell, Daniel Mas Montserrat, Alexander G. Ioannidis
arXiv:2608. 01400v1 Announce Type: new Abstract: Tabular foundation models, driven by in-context learning, have rapidly grown in quality and popularity.
By Rasa Hosseinzadeh, Alex Labach, Zexin Xue, Shuyi Han, Valentin Thomas, Anthony L. Caterini
Tydra is a hybrid Transformer‑State Space Model that interleaves attention and SSM layers for tabular in‑context learning. It achieves a 30% reduction in inference time compared to the Transformer‑only TabPFN while preserving most of its predictive performance. On 30 OpenML datasets, Tydra also outperforms a Hydra model that is roughly ten times larger, demonstrating that hybrid architectures can balance accuracy and efficiency for tabular foundation models.
By Mieszko Komisarczyk, Saurabh Mathur, Maurice Kraus, Sriraam Natarajan, Kristian Kersting
arXiv:2608. 10837v1 Announce Type: cross Abstract: The strong performance of foundation models for tabular tasks comes at substantial inference costs.
By Mykhailo Koshil, Matthias Feurer, Katharina Eggensperger
arXiv:2608. 14472v1 Announce Type: cross Abstract: Neural Architecture Search (NAS) aims to automate neural network architecture design, reducing reliance on human expertise.
By Abhishek Shukla, Ankur Sinha, Faiz Hamid
arXiv:2606. 07345v1 Announce Type: new Abstract: Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples.
By Si-Yang Liu, Han-Jia Ye
TabICLv2 is a new state‑of‑the‑art tabular foundation model that outperforms existing methods on regression and classification tasks. It relies on a synthetic data generation engine for diverse pretraining, architectural innovations such as a scalable softmax attention, and optimized training protocols that replace AdamW with the Muon optimizer. On the TabArena and TALENT benchmarks, TabICLv2 surpasses the current best model, RealTabPFN‑2.5, without any tuning, while also being faster and capable of handling million‑scale datasets with limited GPU memory.
By Jingang Qu, David Holzm\"uller, Ga\"el Varoquaux, Marine Le Morvan
arXiv:2607. 05380v1 Announce Type: new Abstract: In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures.
By Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, Artem Babenko
arXiv:2607. 27546v1 Announce Type: new Abstract: Tabular Foundation Models, such as TabPFN, have received a large amount of recent attention due to their performance on in-context tabular machine learning tasks, which often exceeds classical baselines.
By Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon, Anna Leontjeva, Simon Lucey
arXiv:2603. 15106v2 Announce Type: replace Abstract: Enabling efficient deep neural network (DNN) inference on edge devices with different hardware constraints is a challenging task that typically requires DNN architectures to be specialized for each device separately.
By Mark Deutel, Simon Geis, Axel Plinge