arXiv:2606. 04581v1 Announce Type: cross Abstract: Speculative inference (SPIN) was originally developed as an efficient architecture to accelerate Large Language Models (LLMs).
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2607. 05147v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification.
arXiv:2608.05926v2 Announce Type: replace-cross Abstract: Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM infer...
arXiv:2512. 22420v5 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates LLM inference by verifying draft tokens in parallel.
arXiv:2608. 10362v1 Announce Type: cross Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to speculate multiple tokens, reducing expensive target model decoding steps.
The paper introduces X-CoSD, a communication‑efficient framework for collaborative speculative decoding that allows a small on‑device language model to draft tokens while a server‑side large language model verifies them, even when the two models use different vocabularies. X-CoSD employs hybrid resampling to limit distribution exchange to only the common vocabulary, and its enhanced variant X-CoSD‑E further reduces communication by sending only server‑sampled replacement candidates for device verification. Both variants preserve the server model’s distribution and, according to experiments, accelerate token generation without sacrificing quality.
arXiv:2609.24197v1 Announce Type: new Abstract: Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification b...