The paper introduces Agentic In-Context Learning (Agentic ICL), a framework where fixed‑parameter Transformers learn from interactions that shape the context they receive. It models an interactive learner with state, update, and decision rules, covering active learning, bandits, and online reinforcement learning, and demonstrates that causal Transformers can approximate these algorithms over a finite horizon. Experiments show that Transformers can learn to execute recursions and that different reward signals produce distinct decision policies, with alignment between process supervision and outcome‑optimal actions influencing learned behavior.
Effective machine learning depends not only on how we model data, but also on what data we choose to collect. While large sequence models have revolutionized data modeling, the problem of automated data selection, or "intrinsic curiosity", remains a significant challenge.
arXiv:2606. 19476v1 Announce Type: cross Abstract: Effective machine learning depends not only on how we model data, but also on what data we choose to collect.
LLM agents increasingly improve at inference time by accumulating experience in context rather than by updating parameters. This process is often described as in-context reinforcement learning (ICRL)....
arXiv:2606. 27136v1 Announce Type: new Abstract: For LLM agents in multi-step interactive environments, a key challenge is to make effective use of accumulated interaction experience.
arXiv:2606. 00183v1 Announce Type: cross Abstract: Tree search is a central abstraction behind many language-agent reasoning and decision-making tasks: agents must explore actions, remember failures, and backtrack toward promising alternatives.
arXiv:2606. 17680v1 Announce Type: new Abstract: Reinforcement learning (RL) has emerged as a powerful paradigm for training Large Language Models (LLMs) as agents.
arXiv:2607. 11906v1 Announce Type: new Abstract: The development of decision-pretrained transformers, algorithm distillation, long-context meta-RL, and retrieval-augmented agents has renewed interest in in-context reinforcement learning (ICRL): the ability of a pretrained or fine-tuned decision model to infer latent task rules and improve future behavior from interaction context, without test-time parameter updates.
The paper investigates whether large language models (LLMs) can learn from rewards presented in context, a process termed in‑context reinforcement learning (ICRL). Through experiments on four benchmarks and six models, the authors find that while LLMs read reward signals, altering or removing these signals has little effect on performance, and that trajectory structure—rather than reward content—drives improvement. These results suggest that direct ICRL behaves more like standard in‑context learning (ICL) than true inference‑time reinforcement learning.
The paper investigates whether large language models (LLMs) can learn from rewards presented in context, a process termed in-context reinforcement learning (ICRL). Experiments across four benchmarks and six models show that while LLMs read reward signals, altering or removing these rewards has little impact on performance, and shuffled or corrupted trajectories are as effective as real ones. These findings suggest that direct ICRL behaves more like in-context learning (ICL) than true inference-time reinforcement learning.
The paper introduces Rollout-Time Intervention (RTI), a method that uses reward-side evidence during rollouts to transform unsuccessful trajectories into successful ones, thereby providing learning signals for tasks beyond a policy’s intrinsic exploration limits. RTI filters out trajectories with excessive distributional shifts, allowing these off-policy interventions to guide policy optimization effectively. Experiments on five benchmarks show that RTI consistently outperforms existing RL baselines, enabling post-trained policies to approach the performance of stronger large language models and surpass their original capability boundaries.
The paper studies reinforcement learning (RL) for post‑training language models on reasoning tasks, focusing on a hierarchical reward structure where each component becomes relevant only after the previous ones are resolved. It demonstrates that a Transformer‑based actor–critic algorithm that alternates between KL‑regularized policy sampling, critic fitting, and policy updates achieves minimax‑optimal rates in query budget and regularization strength, and is optimal for a fixed number of prompts. In contrast, sampling from a fixed reference distribution, as used in offline reward modeling, only yields a logarithmic regret decay, highlighting the advantage of on‑policy exploration in concentrating on high‑reward regions.
arXiv:2610.00388v1 Announce Type: cross Abstract: Reinforcement learning enables large language model (LLM) agents to learn multi-step behaviors through interaction with their environments. However,...