The paper introduces Trajectory-Local Adaptive Retrieval (TLAR), a method that leverages a model’s own generated history to improve speculative decoding. TLAR retrieves continuations that closely match the current trajectory and adapts retrieval activation and candidate width based on recent verification outcomes, integrating retrieved and model-generated drafts into a shared candidate tree. Experiments on code debugging, mathematics, and open-ended writing show that TLAR enhances token acceptance under matched verification budgets and boosts overall throughput compared to a draft-model baseline.
Speculative decoding accelerates generation by verifying multiple drafted tokens in a single target-model forward pass, reducing sequential decoding iterations. Model-free variants avoid auxiliary draft models by reusing text and model states already available during generation, but their speedup depends on the reliability of the constructed drafts.
Draft-OPD introduces an on‑policy distillation method for speculative draft models, addressing the mismatch between supervised fine‑tuning and inference by letting the target model supervise the drafter on draft‑induced states. The approach uses target‑assisted rollouts for stable continuations and replays drafting from error positions exposed during verification, enabling the drafter to learn from both accepted and rejected proposals. Experiments demonstrate that Draft‑OPD achieves more than five‑fold lossless acceleration across diverse tasks, outperforming prior draft models such as EAGLE‑3 and DFlash by 23 % and 13 % respectively.
arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...
arXiv:2603. 12617v2 Announce Type: replace-cross Abstract: Speculative decoding has emerged as a widely adopted paradigm for accelerating large language model inference, where a lightweight draft model rapidly generates candidate tokens that are then verified in parallel by a larger target model.
arXiv:2608.20359v1 Announce Type: new Abstract: Large language models (LLMs) are deployed for increasingly complex tasks involving planning and multi-step decision making, but high-quality performanc...
TraceSpec is a framework that links trajectory evaluation with adaptive draft training to improve speculative decoding in large language model agents. It introduces a dynamic execution strategy combined with fixed‑prefix replay and provides a new corpus (TraceSpec‑Traj) of 10,000 training and 2,500 test states covering browsing and software engineering tasks. Experiments show that TraceSpec’s state‑conditioned draft allocation boosts replay throughput by 12.5%–22.6% and increases accepted length by up to 10.0% compared to fixed‑turn supervision.
arXiv:2609.37029v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context...
arXiv:2609.38510v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusi...
TreeGraft introduces a multi-drafter framework that combines drafters of varying costs to build a shared draft tree for tree-based speculative decoding. The stronger drafter rescues and rescoring candidates from the weaker drafter, while a lightweight scheduler decides when to invoke the stronger drafter to manage cost. Experiments on 10 model pairs and 6 benchmarks show TreeGraft improves over the best single-drafter strategy by an average of 15.1% and up to 26.6%.
arXiv:2610.20503v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden featu...
arXiv:2608.04962v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO) has become a standard approach for improving the reasoning abilities of large language models, but its tr...
Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path.