arXiv Computation and Language
SFAD: Speculative Factuality-Aware Decoding
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 1
Entropy-Aware Token Rejection for Improving Speculative Decoding
arXiv:2512.23765v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronge...
Hugging Face Trending Papers
Sep 17
To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals
The paper introduces SwitchSD, an adaptive framework that treats copying as a latent control signal in large language model decoding. By training lightweight probes on internal representations, SwitchSD accurately detects genuine copy intent (AUC > 0.99) and dynamically switches between neural drafting and context-based copying. Experiments on Llama and Qwen models show up to 15 % throughput gains over state‑of‑the‑art baselines such as EAGLE3.
arXiv Computation and Language
1d ago
DLite: History-free Speculative Decoding for Agentic AI
DLite is a history‑free speculative decoding framework designed for agentic AI workloads. It bypasses the need for a historical KV cache by reusing contextual hidden states from the target model and performing local speculative predictions, supplemented by a lightweight sequential head to maintain causal dependencies. Experiments demonstrate that DLite improves acceptance length by 2–6× and outperforms the state‑of‑the‑art DSpark by up to 70%.
arXiv Computation and Language
Sep 18
To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals
The paper introduces SwitchSD, an adaptive framework that controls speculative decoding by distinguishing genuine copy intent from accidental repetitions using lightweight probes on a model’s internal representations. SwitchSD dynamically switches between neural drafting and context-based copying, achieving up to 15% throughput gains over existing baselines such as EAGLE3. The approach turns copying from a noisy heuristic into a principled, model-aware decoding regime.
arXiv Computation and Language
Sep 2
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
Large Language Models suffer from high inference latency, especially when generating long chains of thought. Existing speculative decoding methods draft and verify tokens in parallel but ignore semantic equivalence, causing inefficient rejections. The proposed SemanticSpec framework verifies entire semantic sequences by probing internal hidden states, achieving up to 2.7× speedup on DeepSeekR1-32B and 2.1× on QwQ-32B while outperforming token‑level and sequence‑level baselines in both efficiency and effectiveness.
arXiv AI
Sep 15
Look Before You Leap: Factual Decoding with Internal Attribution Signals
The paper introduces DescaPE, a decoding framework that uses internal model signals to reduce hallucinations in large language models. By identifying a factual‑salient layer span and training a lightweight probe to approximate its signal, DescaPE penalizes high‑risk continuations and rewards factually grounded ones during inference. Experiments on five factuality benchmarks across three LLMs show that DescaPE improves factuality with only a 1.10× latency overhead.