Transformers can learn broad families of tasks during pretraining and adapt to unseen tasks from a short prompt, but a rigorous understanding of this capability is limited. This paper studies how shared cross‑task structure influences the sample complexity of in‑context learning (ICL) by characterizing task‑space complexity through covering numbers, yielding a set of anchor functions that localize unseen tasks and predict responses. The authors construct a Transformer with Softmax attention to approximate this procedure and derive an error bound that separates the effects of pretraining tasks and prompt length, showing that once enough tasks are available the dependence on prompt length becomes dimension‑free.
By Zhongjie Shi, Rongjie Lai, Alexander Cloninger, Wenjing Liao
arXiv:2610.18057v1 Announce Type: cross
Abstract: Transformers can implement reinforcement-learning algorithms in context, yet how off-policy control emerges within Transformers remains poorly unders...
By Yingjun Mo, Yajie Zhao, Fang Kong
The paper studies reinforcement learning (RL) for post‑training language models on reasoning tasks, focusing on a hierarchical reward structure where each component becomes relevant only after the previous ones are resolved. It demonstrates that a Transformer‑based actor–critic algorithm that alternates between KL‑regularized policy sampling, critic fitting, and policy updates achieves minimax‑optimal rates in query budget and regularization strength, and is optimal for a fixed number of prompts. In contrast, sampling from a fixed reference distribution, as used in offline reward modeling, only yields a logarithmic regret decay, highlighting the advantage of on‑policy exploration in concentrating on high‑reward regions.
By Naoki Nishikawa, Taiji Suzuki
arXiv:2609.27869v1 Announce Type: new
Abstract: Long-horizon multimodal agents rely on specialized capabilities for perception, retrieval, reasoning, verification, and execution. Existing designs typ...
By Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Shuo Yang, Edith Cheuk-Han Ngai
arXiv:2603. 06957v2 Announce Type: replace-cross Abstract: We study post-training linear autoregressive models with outcome and process rewards.
By Alireza Mousavi-Hosseini, Murat A. Erdogdu
The paper introduces Self‑Sparse Speculative Rollouts (S$^3$R), a method that uses sparse attention to propose tokens and then verifies them with full attention, thereby reducing KV‑cache traffic while preserving the dense‑policy distribution for reinforcement learning rollouts. S$^3$R eliminates the need for a separate drafter by sharing policy parameters between drafting and verification, and a KV‑aware scheduler limits cache thrashing to maintain parallelism. Experiments on long‑context mathematical reasoning with the Qwen3 model show that S$^3$R achieves 1.35–1.39× higher end‑to‑end training throughput with comparable task performance.
By Yuhang He, Hongyu Ge, Zihan Lin, Siyi Liu, Yongqi Zhang
The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.
By Nusrat Jahan Lia, Aritra Mazumder
The paper introduces Task Specialization Fine-Tuning (TSFT), an online framework that allocates a limited fine‑tuning budget across multiple task regions in Contextual Reinforcement Learning. TSFT predicts fine‑tuning performance with a simple parametric model and solves the budget allocation problem exactly using integer linear programming. Experiments on combinatorial optimization, continuous control, and LLM fine‑tuning show that TSFT outperforms baselines in task coverage and approaches oracle performance.
By Jianan Zhou, Jung-Hoon Cho, Tianyue Zhou, Han Zheng, Jie Zhang, Roy Dong, Yining Ma, Cathy Wu
arXiv:2603. 27044v3 Announce Type: replace-cross Abstract: Deep Reinforcement Learning (DRL) is widely recognized as sample-inefficient, a limitation attributable in part to the high dimensionality and substantial functional redundancy inherent to the policy parameter space.
By Andrea Fraschini, Davide Tenedini, Riccardo Zamboni, Mirco Mutti, Marcello Restelli
arXiv:2608. 07922v1 Announce Type: new Abstract: Adaptive learning needs both a state that preserves what observations imply and opportunities to act on that state.
By Zicheng Lyu, Zengfeng Huang
The paper presents a convergence framework for deep $V$‑learning over a finite horizon $H$, deriving explicit bounds on policy loss by decomposing the Bellman update error into six residuals. It shows how $L^s$ concentrability controls expected $L^1$ loss, quantifies the impact of shared sampling across horizon levels, and provides optimal and near‑optimal sample allocations for statistical error rates. The work also establishes sharp action‑gap bounds under a margin condition, transfers optimal‑gap results to frozen‑iterate gaps, and offers consistency guarantees for generative‑reset approximate‑ERM procedures with exact action scores.
By Yury Kolomeytsev
arXiv:2609.36945v1 Announce Type: new
Abstract: We study the learning dynamics of fine-tuning a policy model on self-generated and reward-weighted data, with particular focus on a generalized version...
By Zhiwei Wang, Yanxi Chen, Yaliang Li, Bolin Ding