TILE (Table-Interfaced LLM Embeddings) bridges large language models (LLMs) and table foundation models (TFMs) to improve tabular learning with string columns. It introduces four key techniques—row serialization, intermediate-layer extraction, columnar pooling, and by-pass features—to extract effective representations from LLMs for tabular data. When combined with TFMs, TILE achieves state‑of‑the‑art performance, outperforming the strongest baseline on 90% of splits across 100 datasets and ranking first on STRABLE’s 108 datasets.
arXiv:2605.12292v2 Announce Type: replace Abstract: Benchmarking tabular learning has revealed the benefit of dedicated architectures, pushing the state of the art. But real-world tables often contai...
arXiv:2606. 04876v1 Announce Type: new Abstract: Tabular foundation models, such as TabPFN, achieve strong performance on tabular datasets with numerical and categorical data, but do not natively handle high-cardinality text features.
arXiv:2606. 15314v1 Announce Type: cross Abstract: Industrial retrofit planning depends on structured operational data rather than free text: planners must estimate whether a newly registered prototype will require a retrofit, which retrofit package it will need, and how long the work will take.
The paper introduces TEmBed, a unified benchmark for evaluating tabular embeddings across four representation levels—cell, row, column, and table—using a diverse set of models. It demonstrates that the best model depends on the specific task and representation level, providing practical guidance for selecting embeddings in real-world applications. The study aims to facilitate the development of more general-purpose tabular representation models.
TabMF is a framework that repurposes pretrained tabular foundation models (TFMs) as encoders for supervised datasets, adding a dataset-level readout and fine‑tuning via contrastive post‑training on synthetic views. The authors instantiate TabMF on TabICLv2 and TabPFN‑2.5 and evaluate the resulting representations on six tasks involving unseen real datasets. The approach achieves the best aggregate performance among both learned and classical dataset representations, excelling in dataset retrieval, compact benchmark construction, and distribution‑shift severity assessment.
arXiv:2608. 02412v1 Announce Type: new Abstract: Large language models (LLMs) have become the default tool for a remarkable range of tasks, yet they have had conspicuously little success at one of the most common machine learning workloads: predictive analytics over tabular data.
arXiv:2606. 09323v1 Announce Type: new Abstract: Tabular encoders are usually evaluated inside task-specific end-to-end pipelines, so models from different training paradigms are difficult to compare directly even when they operate on similar tabular signals.
The paper shows that Tabular Foundation Models (TFMs) are highly overparameterized, with up to 96% of transformer blocks removable without significant loss in performance. Using a closed‑form, label‑free affine translator, the authors demonstrate that halving the depth of eight leading TFMs costs at most $0.002$ median AUROC, while a single‑block TabPFN‑3 remains competitive. Removing depth saves up to 1.5 B parameters, speeds GPU inference up to 6×, and reduces memory usage by 6× with negligible accuracy loss.
arXiv:2607. 24130v1 Announce Type: cross Abstract: Tabular data is the dominant structured-data modality, and learning table representations has become a core research direction.
The study shows that tabular foundation models (TFMs) can be dramatically simplified without losing performance. By replacing up to 96% of transformer blocks with a label‑free affine translator, researchers found that half‑depth models still outperform full‑depth TFMs and tuned GBDTs across 171 classification datasets. The simplification reduces parameters by up to 1.5 B, speeds GPU inference up to 6×, and cuts memory usage by 6× with minimal impact on AUROC.
arXiv:2609.37989v1 Announce Type: new Abstract: Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, ta...
Support-Compiled Feature Folding (SCFF) is a training‑free inference framework that addresses the feature‑side scaling dilemma in tabular foundation models by routing support‑ranked features through bounded leaves of the native encoder, checking residual evidence, and merging encoded messages for a single contextual prediction. This approach transforms quadratic pairwise mixing into linear‑in‑width work with a bounded local working set, achieving dataset‑macro accuracy and NLL improvements across six backbones on an 18‑dataset wide‑table slice. SCFF delivers significant GPU‑memory savings (median 2.09×–2.36×) and, when constrained by a peak‑memory ceiling, further boosts accuracy by up to 4.06 points over the widest single‑leaf baseline. whyItMatters":"SCFF demonstrates that memory‑efficient inference can simultaneously improve accuracy and reduce resource usage in tabular foundation models, offering a practical solution for deploying these models at scale."