arXiv Machine Learning August 5, 2026 By Junxiong Wang, Fengxiang Bie, Jisen Li, Zhongzhu Zhou, Zelei Shao, Yubo Wang, Yinghui Liu, Qingyang Wu, Avner May, Sri Yanamandra, Ce Zhang, Tri Dao, Percy Liang, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu

When RL Meets Adaptive Speculative Training: A Unified Training-Serving System

Read the original on arXiv Machine Learning →

arXiv:2602. 06932v5 Announce Type: replace Abstract: Speculative decoding can significantly accelerate LLM serving, yet most deployments today disentangle speculator training from serving, treating speculator training as a standalone offline modeling problem.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 18

EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts

arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.

By Minseo Kim, Minjae Lee, Seunghyuk Oh, Kevin Galim, Donghoon Kim, Coleman Hooper, Harman Singh, Amir Gholami, Hyung Il Koo, Wonjun Kang

llms agents reinforcement-learning efficiency

arXiv AI

Jun 16

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving

arXiv:2512. 22420v5 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates LLM inference by verifying draft tokens in parallel.

By Rui Li, Zhaoning Zhang, Libo Zhang, Huaimin Wang, Xiang Fu, Zhiquan Lai

llms efficiency benchmarks

arXiv AI

Sep 25

TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference

TIDE (Temporal Incremental Draft Engine) is a serving‑engine‑native framework that integrates online draft adaptation into high‑performance LLM inference. By reusing intermediate hidden states from the target model as training signals, TIDE avoids extra target model computation and serving‑time overhead, activating speculation and draft training only when beneficial. On heterogeneous GPU clusters, TIDE achieves up to 1.66× higher throughput than no‑speculation baselines, reduces training time by up to 3.02×, cuts storage needs by 24×, and improves system throughput by up to 1.22×.

By Jiyoung Park, Hankyu Jang, Changseok Song, Wookeun Jung

llms efficiency

arXiv Machine Learning

Jul 7

AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

arXiv:2607. 03876v1 Announce Type: new Abstract: With the rise of small quantized GGUF-based language models and their increasing use for on-device inference tasks, we have seen the growing need for an approach capable of reliably delivering these models at scale even under severe memory bandwidth constraints such as those imposed by pure CPU implementations.

By Sadra Saremi

llms agents reinforcement-learning efficiency

arXiv Machine Learning

Aug 6

SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts

arXiv:2608. 04962v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training improves the reasoning capabilities of large language models, but autoregressive rollout generation remains a major efficiency bottleneck.

By Nhat Minh Pham, Duy Tung Doan, Thi Duyen Ngo, Vinh Van Nguyen, Khac-Hoai Nam Bui

llms reinforcement-learning

arXiv AI

Jun 16

Frontier: Towards Comprehensive and Accurate LLM Inference Simulation

arXiv:2605. 21312v2 Announce Type: replace-cross Abstract: Modern LLM serving is no longer homogeneous or monolithic.

By Yicheng Feng, Xin Tan, Yangtao Deng, Yimin Jiang, Yibo Zhu, Hong Xu

llms agents reinforcement-learning efficiency benchmarks

One email a morning, machine-written

Related stories

EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving

TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference

AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts

Frontier: Towards Comprehensive and Accurate LLM Inference Simulation