arXiv Machine Learning

Context-Constrained Transfer Learning for Tabular Foundation Models via Data Distillation

arXiv:2607. 04809v1 Announce Type: cross Abstract: Tabular Foundation Models (TFMs) have demonstrated strong empirical performance as black-box inference engines through in-context learning.

arXiv Machine Learning

Aug 31

Generalized Context in Cross Attention for Transfer Learning of Disjoint Tabular Data

The paper introduces CATTLE, a transfer learning framework for disjoint tabular datasets that eliminates the need for shared features by leveraging generalized context learned through transformer projection weights. By using key, value, and query weights from source and target domains, CATTLE performs cross‑domain attention transfer in a data‑agnostic manner. Experiments on ten source‑target pairs demonstrate that CATTLE outperforms nine state‑of‑the‑art baselines, achieving the best average rank (2.9) and a 3.7% AUROC improvement.

By Kazi F. Akhter, Ibna Kowsar, Manar D. Samad

llms fine-tuning benchmarks

arXiv AI

5d ago

TabGift: A Context-Efficient Foundation Model for Tabular Data Generation

arXiv:2610.16990v1 Announce Type: cross Abstract: Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. T...

By Yu-Sheng Li, Jun-Peng Jiang, Han-Jia Ye

llms diffusion data-engineering

arXiv Machine Learning

Oct 3

Distillation of Tabular Foundation Models into Efficient Predictors

The paper presents a method for distilling tabular foundation models (TFMs) into lightweight, dataset‑specific students. By using the full labeled training set as teacher context and training students on both observed and synthetic queries, the authors achieve significant performance gains over traditional supervised models on TabArena and TALENT benchmarks. The distilled students also provide substantial inference speedups, reducing the cost of repeated inference.

By Minho Jeong, Dooho Lee, Jinmo Lee, Jaemin Yoo

llms efficiency

arXiv AI

Aug 12

TACTICL: Task-Aware Compression of Tabular ICL Models

arXiv:2608. 10837v1 Announce Type: cross Abstract: The strong performance of foundation models for tabular tasks comes at substantial inference costs.

By Mykhailo Koshil, Matthias Feurer, Katharina Eggensperger

llms benchmarks

arXiv Machine Learning

Oct 7

TAFFY: A Task-Adaptive Tabular Foundation Model with In-Context Diversity

arXiv:2610.07559v1 Announce Type: new Abstract: Recent progress in tabular foundation models suggests that training on synthetic tasks can substantially improve in-context learning capabilities, with...

By Zijian Li, Xiangchen Song, Gongxu Luo, Jie Qiao, Ruichu Cai, Zhenhao Chen, Xinshuai Dong, Fan Feng, Guangyi Chen, Kun Zhang

llms benchmarks

arXiv Machine Learning

Jun 9

In-Context Learning for Latent Space Bayesian Optimization

arXiv:2606. 09664v1 Announce Type: new Abstract: Bayesian optimization (BO) is a central tool for sample-efficient design, and latent-space Bayesian optimization (LSBO) extends it to structured objects such as molecules and proteins.

By Tuan A. Vu, Harri L\"ahdesm\"aki, Julien Martinelli

llms diffusion benchmarks

arXiv AI

Sep 15

Generalization Can Emerge in Tabular Foundation Models From a Single Table

The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.

By Junwei Ma, Nour Shaheen, Alex Labach, Amine Mhedhbi, Frank Hutter, Anthony L. Caterini, Valentin Thomas

llms benchmarks

arXiv AI

Oct 7

TICDA: Tabular In-Context Data Attribution

arXiv:2610.07996v1 Announce Type: cross Abstract: Tabular foundation models (TFMs) achieve strong predictive performance by conditioning on labeled demonstrations provided in context, without any par...

By Yacine Benihaddadene, Milan Bhan, Eliot Dugelay, Mohammed Jawhar, Benjamin Wong, Nicolas Chesneau, Duong Nguyen

llms rag

arXiv AI

Jun 18

Bounded Context Management for Tabular Foundation Models on Stream Learning

arXiv:2606. 18677v1 Announce Type: cross Abstract: Tabular stream learning requires predictions on sequentially arriving examples under distribution shift.

By Jinmo Lee, Doyun Choi, Moongi Choi, Jaemin Yoo

arXiv AI

5d ago

TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks

arXiv:2610.15699v1 Announce Type: cross Abstract: Generative modeling for tabular data has advanced to the point where models can synthesize realistic records, recover missing values, and generate sa...

By Jayoung Kim, Sanghyun Hong, Noseong Park

llms diffusion

arXiv Machine Learning

Aug 4

TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction

arXiv:2608. 01400v1 Announce Type: new Abstract: Tabular foundation models, driven by in-context learning, have rapidly grown in quality and popularity.

By Rasa Hosseinzadeh, Alex Labach, Zexin Xue, Shuyi Han, Valentin Thomas, Anthony L. Caterini

llms efficiency

arXiv Machine Learning

Jun 9

Hierarchical Projection for Adaptive Knowledge Transfer

arXiv:2606. 08691v1 Announce Type: new Abstract: Modern data-driven applications increasingly involve learning from multiple heterogeneous sources, where a target dataset is limited but related information is available across domains.

By Samhita Pal, Tian Gu

fine-tuning safety

Related stories