arXiv Machine Learning
Provable Transformer Context Compression for In-Context Reinforcement Learning
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 25
Transformers as Cross-Task Learners: Shared Structure Drives Sample Efficiency in In-Context Learning
Transformers can learn broad families of tasks during pretraining and adapt to unseen tasks from a short prompt, but a rigorous understanding of this capability is limited. This paper studies how shared cross‑task structure influences the sample complexity of in‑context learning (ICL) by characterizing task‑space complexity through covering numbers, yielding a set of anchor functions that localize unseen tasks and predict responses. The authors construct a Transformer with Softmax attention to approximate this procedure and derive an error bound that separates the effects of pretraining tasks and prompt length, showing that once enough tasks are available the dependence on prompt length becomes dimension‑free.
arXiv AI
Oct 15
From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-Learning
arXiv:2610.18057v1 Announce Type: cross Abstract: Transformers can implement reinforcement-learning algorithms in context, yet how off-policy control emerges within Transformers remains poorly unders...
arXiv Machine Learning
Oct 7
Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
The paper studies reinforcement learning (RL) for post‑training language models on reasoning tasks, focusing on a hierarchical reward structure where each component becomes relevant only after the previous ones are resolved. It demonstrates that a Transformer‑based actor–critic algorithm that alternates between KL‑regularized policy sampling, critic fitting, and policy updates achieves minimax‑optimal rates in query budget and regularization strength, and is optimal for a fixed number of prompts. In contrast, sampling from a fixed reference distribution, as used in offline reward modeling, only yields a logarithmic regret decay, highlighting the advantage of on‑policy exploration in concentrating on high‑reward regions.
arXiv AI
Sep 24
Learning What to Activate: Combinatorial Capability Allocation for Long-Horizon Multimodal Agents
arXiv:2609.27869v1 Announce Type: new Abstract: Long-horizon multimodal agents rely on specialized capabilities for perception, retrieval, reasoning, verification, and execution. Existing designs typ...
arXiv AI
Aug 13
Post-Training with Policy Gradients: Optimality and the Base Model Barrier
arXiv:2603. 06957v2 Announce Type: replace-cross Abstract: We study post-training linear autoregressive models with outcome and process rewards.
arXiv Machine Learning
Oct 12
S$^3$R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding
The paper introduces Self‑Sparse Speculative Rollouts (S$^3$R), a method that uses sparse attention to propose tokens and then verifies them with full attention, thereby reducing KV‑cache traffic while preserving the dense‑policy distribution for reinforcement learning rollouts. S$^3$R eliminates the need for a separate drafter by sharing policy parameters between drafting and verification, and a KV‑aware scheduler limits cache thrashing to maintain parallelism. Experiments on long‑context mathematical reasoning with the Qwen3 model show that S$^3$R achieves 1.35–1.39× higher end‑to‑end training throughput with comparable task performance.