The paper introduces CATTLE, a transfer learning framework for disjoint tabular datasets that eliminates the need for shared features by leveraging generalized context learned through transformer projection weights. By using key, value, and query weights from source and target domains, CATTLE performs cross‑domain attention transfer in a data‑agnostic manner. Experiments on ten source‑target pairs demonstrate that CATTLE outperforms nine state‑of‑the‑art baselines, achieving the best average rank (2.9) and a 3.7% AUROC improvement.
arXiv:2603. 07523v3 Announce Type: replace Abstract: Transferring knowledge by fine-tuning large-scale pre-trained networks has become a standard paradigm for downstream tasks, yet the knowledge of a pre-trained model is tightly coupled with monolithic architecture, which restricts flexible reuse across models of varying scales.
arXiv:2607. 17467v1 Announce Type: cross Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples.
arXiv:2607. 00479v1 Announce Type: new Abstract: Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.
SOMTab is a Set-Order Mamba architecture designed for efficient tabular in-context learning. It separates representation construction from query-conditioned retrieval, using Mamba-based state‑space mixing to build compact row and column representations while retaining attention for final prediction. The model, along with a synthetic prior called DCH‑TailMix, achieves performance comparable to Transformer‑based tabular foundation models but with faster inference and lower GPU memory usage.
arXiv:2608. 09091v1 Announce Type: cross Abstract: Transfer learning is particularly useful in settings with limited training data, and within image classification it is common to transfer learn upon massive datasets like ImageNet , CIFAR-100, or COCO .
The paper investigates whether deep transformer layers require context from the residual stream to compute value vectors. It finds that allowing deeper layers to use a context‑free value vector—preserving original token information—significantly improves performance, and adding context afterward yields little extra benefit. The authors introduce Bank of Values (BoV), a lookup table of token‑specific value vectors for the last third of layers, which reduces compute and memory while matching or surpassing prior methods on large models.
The paper demonstrates that a tabular foundation model can achieve strong generalization using only a single real table for self‑supervised pre‑training, challenging the belief that large synthetic or real datasets are necessary. By systematically pre‑training and evaluating across diverse benchmarks, the authors show that the number and quality of tasks that can be derived from a dataset are critical for downstream performance. This finding suggests that carefully constructed task sets from limited data can enable effective transfer learning in tabular models.
The paper evaluates data‑efficient finetuning methods for vision foundation models, focusing on Visual Prompting (VP) and comparing it with Linear Probing, Full Fine‑tuning, and Low Rank Adaptation across different model families (Attention, Convolution, Mamba). It provides an empirical overview of how these models and transfer paradigms interact, broadening previous studies by including more architectures and tuning hyperparameters. The authors also investigate regularization techniques to enhance VP performance, aiming to offer practical guidelines for users.
The paper evaluates data‑efficient finetuning methods for vision foundation models, focusing on Visual Prompting (VP) and comparing it with Linear Probing, Full Fine‑tuning, and Low Rank Adaptation across Attention, Convolution, and Mamba model families. It expands on prior work by testing a broader set of models, tuning hyperparameters, and applying regularization techniques to improve VP performance. The study aims to provide practical guidelines for users working under resource constraints.
arXiv:2602. 14761v2 Announce Type: replace-cross Abstract: Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability.
arXiv:2607. 13425v1 Announce Type: cross Abstract: Learning effectively from limited data is critical in domains like security where labeled examples are scarce.
The paper investigates how Tabular Foundation Models (TFMs) can achieve strong transfer learning by self‑supervised pre‑training on a single real table. It finds that a table’s usefulness is largely determined by its number of features rather than instances, and that fine‑grained column‑level preprocessing improves downstream performance while dataset‑level filtering does not. The authors propose that tabular in‑context generalization is primarily retrieval‑based, with models learning to identify and aggregate relevant examples from the provided context.