arXiv Machine Learning July 8, 2026 By Hongkang Li, Songtao Lu, Xiaodong Cui, Pin-Yu Chen, Meng Wang

How Can Mamba Learn In Context with Outliers and Generalize Provably?

Read the original on arXiv Machine Learning →

arXiv:2510. 00399v2 Announce Type: replace Abstract: The Mamba model has gained significant attention for its computational advantages over Transformer-based models, while achieving comparable performance across a wide range of language tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

4d ago

Theory on Attention Dynamics for Out-of-Distribution In-Context Learning

arXiv:2609.36448v1 Announce Type: new Abstract: Transformers have demonstrated remarkable in-context learning (ICL) capabilities, enabling them to perform new tasks without additional fine-tuning. Ho...

By Junze Deng, Daouda Sow, Sen Lin, Yingbin Liang

llms fine-tuning

arXiv Machine Learning

Jul 2

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

arXiv:2607. 00479v1 Announce Type: new Abstract: Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.

By Peilin Liu, Ding-Xuan Zhou

llms

arXiv Machine Learning

Jul 7

Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch

arXiv:2607. 03660v1 Announce Type: cross Abstract: Modern sequence models have a striking capacity for in-context learning (ICL); they can perform new tasks based only on examples given in the prompt.

By Mary Letey, Yue M. Lu, Cengiz Pehlevan, Jacob Zavatone-Veth

llms

arXiv Machine Learning

Jun 8

The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

arXiv:2606. 06814v1 Announce Type: cross Abstract: The transformer's emergent ability to perform in-context learning (ICL) has sparked a wide range of studies designed to understand its underlying mechanisms.

By Soo Min Kwon, Alec S. Xu, Can Yaras, Dogyoon Song, Laura Balzano, Qing Qu

llms

arXiv Machine Learning

Jun 4

Activation-Based Active Learning for In-Context Learning: Challenges and Insights

arXiv:2606. 05134v1 Announce Type: cross Abstract: Deep active learning has previously been explored for LLM in-context sample selection, but not with methods that utilise recent advances in understanding of transformer activations.

By Yaseen M. Osman, Geoff V. Merrett, Stuart E. Middleton

llms

arXiv Machine Learning

Sep 10

Conditioned Initialization for Attention

arXiv:2609.07086v1 Announce Type: new Abstract: Transformers are a dominant architecture in modern machine learning, powering applications across vision, language, and beyond. At the core of their su...

By Hemanth Saratchandran, Simon Lucey

llms safety

One email a morning, machine-written

Related stories

Theory on Attention Dynamics for Out-of-Distribution In-Context Learning

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch

The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

Activation-Based Active Learning for In-Context Learning: Challenges and Insights

Conditioned Initialization for Attention