arXiv Machine Learning June 4, 2026 By Yaseen M. Osman, Geoff V. Merrett, Stuart E. Middleton

Activation-Based Active Learning for In-Context Learning: Challenges and Insights

Read the original on arXiv Machine Learning →

arXiv:2606. 05134v1 Announce Type: cross Abstract: Deep active learning has previously been explored for LLM in-context sample selection, but not with methods that utilise recent advances in understanding of transformer activations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Jul 2

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

arXiv:2607. 00479v1 Announce Type: new Abstract: Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.

By Peilin Liu, Ding-Xuan Zhou

llms

arXiv Machine Learning

Jul 7

Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch

arXiv:2607. 03660v1 Announce Type: cross Abstract: Modern sequence models have a striking capacity for in-context learning (ICL); they can perform new tasks based only on examples given in the prompt.

By Mary Letey, Yue M. Lu, Cengiz Pehlevan, Jacob Zavatone-Veth

llms

arXiv Machine Learning

4d ago

Theory on Attention Dynamics for Out-of-Distribution In-Context Learning

arXiv:2609.36448v1 Announce Type: new Abstract: Transformers have demonstrated remarkable in-context learning (ICL) capabilities, enabling them to perform new tasks without additional fine-tuning. Ho...

By Junze Deng, Daouda Sow, Sen Lin, Yingbin Liang

llms fine-tuning

arXiv Machine Learning

Jul 8

How Can Mamba Learn In Context with Outliers and Generalize Provably?

arXiv:2510. 00399v2 Announce Type: replace Abstract: The Mamba model has gained significant attention for its computational advantages over Transformer-based models, while achieving comparable performance across a wide range of language tasks.

By Hongkang Li, Songtao Lu, Xiaodong Cui, Pin-Yu Chen, Meng Wang

llms fine-tuning

arXiv AI

2d ago

Capturing In-Context Learning Dynamics with Task Operators

arXiv:2610.01054v1 Announce Type: cross Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requi...

By Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang

llms

arXiv Machine Learning

Jun 25

How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off

arXiv:2510. 01163v2 Announce Type: replace Abstract: The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to adapt to new tasks from only a handful of examples.

By Wa\"iss Azizian, Ali Hasan

llms

One email a morning, machine-written

Related stories

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch

Theory on Attention Dynamics for Out-of-Distribution In-Context Learning

How Can Mamba Learn In Context with Outliers and Generalize Provably?

Capturing In-Context Learning Dynamics with Task Operators

How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off