arXiv Machine Learning July 2, 2026 By Peilin Liu, Ding-Xuan Zhou

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

Read the original on arXiv Machine Learning →

arXiv:2607. 00479v1 Announce Type: new Abstract: Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Jun 30

Generalization Analysis of Transformers in Distribution Regression

arXiv:2606. 29256v1 Announce Type: cross Abstract: In recent years, models based on the Transformer architecture have seen widespread applications and have become one of the core tools in the field of deep learning.

By Peilin Liu, Ding-Xuan Zhou

llms nlp fine-tuning

arXiv Machine Learning

Aug 11

Training-Free Universal Approximation by Prompting Random Transformers

arXiv:2608. 09558v1 Announce Type: new Abstract: How expressive is prompting a transformer?

By Alexander Hsu, Rongjie Lai

llms

arXiv Machine Learning

Jul 7

Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch

arXiv:2607. 03660v1 Announce Type: cross Abstract: Modern sequence models have a striking capacity for in-context learning (ICL); they can perform new tasks based only on examples given in the prompt.

By Mary Letey, Yue M. Lu, Cengiz Pehlevan, Jacob Zavatone-Veth

llms

arXiv Machine Learning

Jun 17

Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective

arXiv:2512. 11784v2 Announce Type: replace Abstract: Softmax attention is a central component of transformer architectures, yet its nonlinear structure poses significant challenges for theoretical analysis.

By Etienne Boursier, Claire Boyer

llms

arXiv AI

Jun 16

Adaptive inference and function vectors in deep transformers

arXiv:2606. 16694v1 Announce Type: cross Abstract: Transformers are widely used as a general-purpose substrate for learning complex correlations between a large collection of coupled variables, but their internal mechanisms have remained mysterious.

By Ravin Raj, Gautam Reddy

llms

arXiv AI

Jun 29

Flexformer: Flexible Linear Transformer with Learnable Attention Kernel

arXiv:2606. 27748v1 Announce Type: cross Abstract: Transformer models rely on attention mechanism to capture long-range dependencies but suffer from quadratic complexity, limiting their scalability to long sequences.

By Haoran Zhang, Feng Zhou

llms

Related stories

Generalization Analysis of Transformers in Distribution Regression

Training-Free Universal Approximation by Prompting Random Transformers

Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch

Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective

Adaptive inference and function vectors in deep transformers

Flexformer: Flexible Linear Transformer with Learnable Attention Kernel