arXiv AI September 4, 2026 By Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova, Loris D'Antoni

Grammar-Aligned Decoding

The paper introduces Grammar‑Aligned Decoding (GAD), addressing the issue that conventional grammar‑constrained decoding (GCD) can distort a large language model’s probability distribution, yielding grammatical but low‑likelihood outputs. GAD proposes an adaptive sampling method, Approximate Expected Futures (ASAp), which uses prior samples to over‑approximate future grammaticality, ensuring outputs remain both grammatical and faithful to the model’s conditional probabilities. Experiments on code generation and structured NLP tasks demonstrate that ASAp often produces higher‑likelihood outputs than existing GCD techniques while still enforcing the required grammatical constraints.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 12

The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding

arXiv:2608. 10137v1 Announce Type: cross Abstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step.

By I\c{s}{\i}l \"Ozg\"u, Yaoxuan Wu, Guy Van den Broeck, Miryung Kim

llms efficiency safety

arXiv AI

Jun 4

Constrained Adaptive Rejection Sampling

arXiv:2510. 01902v2 Announce Type: replace Abstract: Language Models (LMs) are increasingly used in applications where generated outputs must satisfy strict semantic or syntactic constraints.

By Pawe{\l} Parys, Sairam Vaidya, Taylor Berg-Kirkpatrick, Loris D'Antoni

llms efficiency

arXiv Computation and Language

Aug 25

Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars

arXiv:2602.00612v3 Announce Type: replace Abstract: Diffusion Large Language Models (dLLMs) have demonstrated promising generative capabilities and are increasingly used to produce formal languages d...

By Yitong Zhang, Yongmin Li, Yuetong Liu, Jia Li, Xiaoran Jia, Zherui Li, Ge Li

llms diffusion benchmarks

arXiv Machine Learning

Jul 9

Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

arXiv:2607. 07026v1 Announce Type: new Abstract: Constrained decoding is essential for serving LLMs, ensuring that generated outputs follow specific structures such as JSON schema-formatted function calls.

By Meihua Dang, Stefano Ermon

llms agents diffusion data-engineering

Hugging Face Trending Papers

Jul 8

Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

Constrained decoding is essential for serving LLMs, ensuring that generated outputs follow specific structures such as JSON schema-formatted function calls. Existing systems are designed for autoregressive models and assume left-to-right generation, masking out invalid next tokens at each step.

llms agents diffusion data-engineering

arXiv AI

Jun 30

The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding

arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.

By Avinash Reddy, Thayne T. Walker, James S. Ide, Amrit Singh Bedi

llms benchmarks

One email a morning, machine-written

Related stories

The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding

Constrained Adaptive Rejection Sampling

Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars

Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding