arXiv:2606. 04581v1 Announce Type: cross Abstract: Speculative inference (SPIN) was originally developed as an efficient architecture to accelerate Large Language Models (LLMs).
arXiv:2607. 05147v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification.
arXiv:2608.05926v2 Announce Type: replace-cross Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM infer...
arXiv:2512. 22420v5 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates LLM inference by verifying draft tokens in parallel.
arXiv:2608. 10362v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to speculate multiple tokens, reducing expensive target model decoding steps.
The paper introduces X-CoSD, a communication‑efficient framework for collaborative speculative decoding that allows a small on‑device language model to draft tokens while a server‑side large language model verifies them, even when the two models use different vocabularies. X-CoSD employs hybrid resampling to limit distribution exchange to only the common vocabulary, and its enhanced variant X-CoSD‑E further reduces communication by sending only server‑sampled replacement candidates for device verification. Both variants preserve the server model’s distribution and, according to experiments, accelerate token generation without sacrificing quality.
arXiv:2609.24197v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification b...
arXiv:2606. 18967v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a representative post-training paradigm for LLMs, enabling strong reasoning and agentic capabilities.
arXiv:2608.22788v1 Announce Type: new Abstract: Large-scale rollouts have become a core component of modern LLM systems, spanning reinforcement learning (RL) post-training, on-policy distillation (OP...
ASPIRE introduces a non‑synchronized batched self‑speculative decoding framework for long‑context LLM inference, addressing the memory bottleneck of attention by drafting tokens with sparse attention and verifying them with full attention. It combines a unified mixed forward pass, a lightweight online speculation scheduler that lets each request independently decide when to verify, and an intra‑draft refresh layer that updates the sparse context at every draft step. Experiments on three models and five benchmarks show 1.70–4.58× speedup over autoregressive baselines and a 27% average improvement over the strongest prior self‑speculative methods.
arXiv:2608. 05926v1 Announce Type: cross Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks.
arXiv:2412. 04504v2 Announce Type: replace-cross Abstract: As large language models (LLMs) grow in popularity for their diverse capabilities, improving the efficiency of their inference systems has become increasingly critical.
arXiv:2609.37532v1 Announce Type: cross Abstract: Growing large language model applications demand efficient inference. At high concurrency, block-diffusion speculative decoding suffers from verifica...