arXiv Computation and Language
Self-Speculation for Faster Reasoning Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 15
Post-Reasoning: Improving the Performance of Non-Thinking Models at No Cost
arXiv:2605.06165v2 Announce Type: replace Abstract: As the widespread adoption of Large Language Models (LLMs) accelerates, token consumption from intermediate reasoning traces increasingly contribut...
arXiv AI
Jun 4
Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time
arXiv:2504. 12329v2 Announce Type: replace-cross Abstract: Recent advances leverage post-training to enhance model reasoning performance, which typically requires costly training pipelines and still suffers from inefficient, overly lengthy outputs.
arXiv Computation and Language
Sep 2
Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
Large Language Models suffer from high inference latency, especially when generating long chains of thought. Existing speculative decoding methods draft and verify tokens in parallel but ignore semantic equivalence, causing inefficient rejections. The proposed SemanticSpec framework verifies entire semantic sequences by probing internal hidden states, achieving up to 2.7× speedup on DeepSeekR1-32B and 2.1× on QwQ-32B while outperforming token‑level and sequence‑level baselines in both efficiency and effectiveness.
arXiv Computation and Language
Oct 8
SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
arXiv:2509.14093v3 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting can substantially improve the reasoning ability of large language models (LLMs), but it often comes with hig...
arXiv AI
Jun 4
Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression
arXiv:2510. 08647v2 Announce Type: replace-cross Abstract: Recent developments have enabled advanced reasoning in Large Language Models (LLMs) via long Chain-of-Thought (CoT), trading efficiency during inference for performance.
arXiv AI
Oct 9
From Chain-of-Thought to Loops: Non-Autoregressive Latent Reasoning via Looped Transformers
The paper introduces LLoCoT, a looped latent‑reasoning framework that replaces the traditional left‑to‑right generation of chain‑of‑thought tokens with iterative refinement of a compact latent workspace using a shared transformer. By jointly updating latent slots over a few iterations and sampling latent tokens in parallel, LLoCoT feeds these into an autoregressive decoder for answer generation. Experiments on HumanEval and MBPP show that LLoCoT matches or surpasses explicit‑CoT baselines while dramatically reducing reasoning latency and increasing throughput.