arXiv AI

The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding

arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.

arXiv Machine Learning

Jul 9

Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

arXiv:2607. 07026v1 Announce Type: new Abstract: Constrained decoding is essential for serving LLMs, ensuring that generated outputs follow specific structures such as JSON schema-formatted function calls.

By Meihua Dang, Stefano Ermon

llms agents diffusion data-engineering

arXiv Computation and Language

Aug 25

Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars

arXiv:2602.00612v3 Announce Type: replace Abstract: Diffusion Large Language Models (dLLMs) have demonstrated promising generative capabilities and are increasingly used to produce formal languages d...

By Yitong Zhang, Yongmin Li, Yuetong Liu, Jia Li, Xiaoran Jia, Zherui Li, Ge Li

llms diffusion benchmarks

arXiv AI

Jul 24

DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

arXiv:2607. 20467v1 Announce Type: new Abstract: While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds.

By Yanhua Jiao, Tianyi Wu, Xiaoxi Sun, Yulin Li, HuiLing Zhen, Libo Qin, Baotian Hu, Zhuotao Tian, Min Zhang

llms diffusion benchmarks

Hugging Face Trending Papers

Jul 8

Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

Constrained decoding is essential for serving LLMs, ensuring that generated outputs follow specific structures such as JSON schema-formatted function calls. Existing systems are designed for autoregressive models and assume left-to-right generation, masking out invalid next tokens at each step.

llms agents diffusion data-engineering

arXiv AI

Sep 1

Entropy-Aware Token Rejection for Improving Speculative Decoding

arXiv:2512.23765v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronge...

By Tiancheng Su, Meicong Zhang, Guoxiu He

llms benchmarks

arXiv Machine Learning

Aug 12

The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding

arXiv:2608. 10137v1 Announce Type: cross Abstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step.

By I\c{s}{\i}l \"Ozg\"u, Yaoxuan Wu, Guy Van den Broeck, Miryung Kim

llms efficiency safety

arXiv AI

Sep 15

Carryover Drafting: Recycling Rejected States for Speculative Decoding

arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept seve...

By Jahyun Koo, Sunghyeon Woo, Jaeeun Kil, Jeongtae Lee, Sungjae Lee, Kyomin Jung, Minsub Kim

llms rag

arXiv AI

Jun 2

EPIC: Efficient and Parallel Inference under CFG Constraints for Diffusion Language Models

arXiv:2606. 00722v1 Announce Type: cross Abstract: Controlling language model outputs is essential for ensuring structural validity, reliability, and downstream usability, and diffusion language models are no exception.

By Hyundong Jin, Yo-Sub Han

llms diffusion benchmarks

arXiv AI

Jul 14

Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

arXiv:2607. 10661v1 Announce Type: cross Abstract: Speculative decoding has significantly accelerated Large Language Model (LLM) inference by alleviating memory-bound bottlenecks.

By Zipeng Gao, Zhi Zheng, Qingrong Xia, Junda Lin, Ziwei Zhao, Tong Xu, Zhefeng Wang, Enhong Chen

llms efficiency benchmarks

arXiv AI

Jun 3

$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction

arXiv:2604. 18995v2 Announce Type: replace-cross Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive generation by enabling parallel token prediction.

By Zhenbang Du, Kejing Xia, Xinrui Zhong, Yonggan Fu, Nicolai Oswald, Binfei Ji, Brucek Khailany, Pavlo Molchanov, Yingyan Lin

llms diffusion fine-tuning efficiency

arXiv Machine Learning

3d ago

SEED: Self-Speculative Decoding via Implicit Encoder-Decoder

arXiv:2609.36590v1 Announce Type: cross Abstract: Self-speculative decoding accelerates large language model (LLM) inference by drafting tokens from the target model itself, but faces a sharp tradeof...

By Hankun Lin, Patrick Pynadath, Ruqi Zhang

llms fine-tuning benchmarks

arXiv AI

Sep 1

DIP: Dynamic In-Context Planner For Diffusion Language Models

arXiv:2601.03199v2 Announce Type: replace-cross Abstract: Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context...

By Yang Li, Han Meng, Chenan Wang, Zhenyu Bi, Xuan Wang, Haipeng Chen

llms diffusion benchmarks

Related stories