arXiv Machine Learning June 30, 2026 By Zhenyu Liu, Huaze Tang, Shao-Lun Huang

A Theoretical Interpretation of In-Context Learning via Probabilistic Modeling

Read the original on arXiv Machine Learning →

arXiv:2606. 28926v1 Announce Type: cross Abstract: In-context learning (ICL) is an emerging paradigm that employs the semantic information inherent in large language models (LLMs) for generating answers to user queries.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Computation and Language

Sep 16

In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

The paper investigates whether in‑context learning (ICL) can replace instruction tuning for multilingual language models, especially as model size varies. It highlights the difficulty of obtaining high‑quality instruction data in multilingual settings and compares the performance of ICL versus instruction‑tuned models. The findings show that a performance gap persists between the two approaches, suggesting the need for further research to close it.

By David Ponce, Thierry Etchegoyhen

llms fine-tuning safety

arXiv Machine Learning

Jun 25

How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off

arXiv:2510. 01163v2 Announce Type: replace Abstract: The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to adapt to new tasks from only a handful of examples.

By Wa\"iss Azizian, Ali Hasan

llms

arXiv AI

Jun 9

Emergence of Context Characteristics Sensitivity in Large Language Models

arXiv:2606. 09525v1 Announce Type: cross Abstract: During instruction fine-tuning (IFT), large language models (LLMs) learn to follow instructions by using the provided context to answer a query.

By Nadya Yuki Wangsajaya, Haeun Yu, Isabelle Augenstein

llms reinforcement-learning fine-tuning

arXiv AI

Jul 7

Context Tuning for In-Context Optimization

arXiv:2507. 04221v3 Announce Type: replace-cross Abstract: We introduce Context Tuning, a simple and effective method to significantly enhance few-shot adaptation of large language models (LLMs) without weight updates.

By Jack Lu, Ryan Teehan, Zhenbang Yang, Mengye Ren

llms benchmarks

arXiv AI

Jun 30

LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction

arXiv:2606. 29407v1 Announce Type: cross Abstract: There has been increasing interest in exploring the capabilities of advanced large language models (LLMs) in the field of information extraction (IE), specifically focusing on tasks related to named entity recognition (NER) and relation extraction (RE).

By Xiao You, Tianwei Yan, Shan Zhao

llms nlp

arXiv AI

Jun 2

Toward Robust In-Context Learning: Leveraging Out-of-distribution Proxies for Target Inaccessible Demonstration Retrieval

arXiv:2606. 00014v1 Announce Type: cross Abstract: Although studies have demonstrated that Large Language Models (LLMs) can perform well on Out-of-Distribution (OOD) tasks, their advantage tends to diminish as the distribution shift becomes more severe.

By Hao Xu, Rite Bo, Fausto Giunchiglia, Yingji Li, Rui Song

llms

One email a morning, machine-written

Related stories

In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off

Emergence of Context Characteristics Sensitivity in Large Language Models

Context Tuning for In-Context Optimization

LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction

Toward Robust In-Context Learning: Leveraging Out-of-distribution Proxies for Target Inaccessible Demonstration Retrieval