arXiv Machine Learning
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jul 17
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
arXiv:2607. 14952v1 Announce Type: new Abstract: A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment.
arXiv Machine Learning
Sep 22
H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache
arXiv:2609.24197v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification b...
arXiv Computation and Language
Sep 18
On-Demand Attention: Language Models Know When to Recall
The paper introduces On‑Demand Attention (ODA), a decoding strategy that lets pretrained language models decide when to use global attention based on a lightweight recall head. ODA keeps the original model weights unchanged, only training the recall head, and can be implemented with GPU‑side conditional execution to reduce global reads. Experiments on Qwen, Gemma, and hybrid‑attention models show that ODA largely recovers performance lost by local attention while cutting the number of global attention operations.
arXiv AI
Jun 2
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding
arXiv:2606. 00144v1 Announce Type: cross Abstract: Speculative decoding speeds up autoregressive decoding by using a drafter to propose multiple tokens that a verifier validates in parallel.
arXiv Machine Learning
Jun 18
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.
arXiv Machine Learning
Aug 6
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts
arXiv:2608. 04962v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training improves the reasoning capabilities of large language models, but autoregressive rollout generation remains a major efficiency bottleneck.