← Back to all news
arXiv Machine Learning By Kazi F. Akhter, Ibna Kowsar, Manar D. Samad

Generalized Context in Cross Attention for Transfer Learning of Disjoint Tabular Data

Read the original on arXiv Machine Learning →

The paper introduces CATTLE, a transfer learning framework for disjoint tabular datasets that eliminates the need for shared features by leveraging generalized context learned through transformer projection weights. By using key, value, and query weights from source and target domains, CATTLE performs cross‑domain attention transfer in a data‑agnostic manner. Experiments on ten source‑target pairs demonstrate that CATTLE outperforms nine state‑of‑the‑art baselines, achieving the best average rank (2.9) and a 3.7% AUROC improvement.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 4

Breaking the Scale Barrier: One-Shot Knowledge Transfer via Frequency Transform

arXiv:2603. 07523v3 Announce Type: replace Abstract: Transferring knowledge by fine-tuning large-scale pre-trained networks has become a standard paradigm for downstream tasks, yet the knowledge of a pre-trained model is tightly coupled with monolithic architecture, which restricts flexible reuse across models of varying scales.

By Jianlu Shen, Fu Feng, Yucheng Xie, Jiaqi Lv, Xin Geng
fine-tuningbenchmarks
More like this →
arXiv Machine Learning
Aug 31

SOMTab: Set-Order Mamba for Efficient Tabular In-Context Learning

SOMTab is a Set-Order Mamba architecture designed for efficient tabular in-context learning. It separates representation construction from query-conditioned retrieval, using Mamba-based state‑space mixing to build compact row and column representations while retaining attention for final prediction. The model, along with a synthetic prior called DCH‑TailMix, achieves performance comparable to Transformer‑based tabular foundation models but with faster inference and lower GPU memory usage.

By Hao Wang, Siyu Zhang, Wei Ma
llmsbenchmarks
More like this →
arXiv Computation and Language
Aug 25

Do Value Vectors in Deep Layers Need Context from the Residual Stream?

The paper investigates whether deep transformer layers require context from the residual stream to compute value vectors. It finds that allowing deeper layers to use a context‑free value vector—preserving original token information—significantly improves performance, and adding context afterward yields little extra benefit. The authors introduce Bank of Values (BoV), a lookup table of token‑specific value vectors for the last third of layers, which reduces compute and memory while matching or surpassing prior methods on large models.

By Muyu He, Yuchen Liu, Qingya Huang, Li Zhang
llmsbenchmarks
More like this →