arXiv Machine Learning By Yu Tian

REAL Loss: Optimizing Real Expected Acceptance Length for Speculative Decoding

Read the original on arXiv Machine Learning →

The paper introduces REAL, a new training loss that directly optimizes the real expected acceptance length (EAL) for speculative decoding in large language models. Unlike previous block‑level surrogates, REAL uses path weights for unbiased EAL estimation and Rao‑Blackwellization to reduce variance. Experiments with a Qwen3.6‑35B target and Domino drafter show that REAL outperforms existing objectives across multiple tasks and seeds, without altering the drafter architecture or inference process.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language
Nov 4

TVP: TV is Almost All You Need for Speculative Decoding

The paper introduces TVP, a loss family that improves speculative decoding for large language models by focusing on expected acceptance length. It shows that TV, the total variation distance, can match or surpass traditional draft-training objectives when properly tuned, and that TVP’s power parameter balances coverage and acceptance. Experiments across multiple drafter architectures and target models demonstrate that TVP yields higher acceptance lengths, outperforming cross‑entropy and existing training recipes by significant margins.

By Aaryam Sharma
arXiv Machine Learning
Jun 2

LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

arXiv:2602. 23881v2 Announce Type: replace Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model.

By Alexander Samarin, Sergei Krutikov, Anton Shevtsov, Sergei Skvortsov, Filipp Fisin, Alexander Golubev
arXiv Machine Learning
Oct 8

Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

The paper introduces the Expected Decoding Rounds (EDR) objective, a theoretical framework that models speculative decoding as a Markov reward process and directly optimizes the number of decoding rounds. EDR weights local rejection costs by state occupancies, eliminating auxiliary hyperparameters and enabling unbiased stochastic optimization via temporal‑difference gradients. Applying EDR to fine‑tune state‑of‑the‑art draft models (DSpark and DFly) improves mean accepted length and outperforms prior training objectives across nine benchmarks in math reasoning, code generation, and chat.

By Yunxiao Zhao, Changxiao Cai