arXiv Computation and Language
Suffix-Constrained Greedy Search Algorithms for Causal Language Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Jun 16
Simplifying the Modeling of Arbitrary Conditionals in Natural Language
arXiv:2606. 14943v1 Announce Type: cross Abstract: Causal Transformers model sequences through an autoregressive factorization of the joint distribution, which enables efficient left-to-right decoding and conditional likelihood computation.
arXiv Computation and Language
Aug 25
Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars
arXiv:2602.00612v3 Announce Type: replace Abstract: Diffusion Large Language Models (dLLMs) have demonstrated promising generative capabilities and are increasingly used to produce formal languages d...
arXiv AI
Jul 24
Tractable Hierarchical Control of Autoregressive Language Models
arXiv:2607. 20483v1 Announce Type: new Abstract: Constraining the generation of autoregressive large language models (LLMs) is an important component of integrating language models into formal systems.
arXiv AI
Sep 4
RECAST: Expanding the Boundaries of LLMs' Complex Instruction Following with Multi-Constraint Data
RECAST is a new framework that generates datasets with far more constraints per example than existing benchmarks, aiming to push large language models (LLMs) to better follow complex instructions. The authors built RECAST-30K, a 30,000‑instance dataset covering 19 constraint types extracted from real prompt‑response pairs, and showed that fine‑tuning on it improves LLMs’ ability to handle complex tasks without harming general performance. RECAST also provides rule‑based and LLM‑based validators for automatic constraint verification, enabling reward‑based reinforcement learning to further enhance model performance on challenging tasks.
arXiv AI
Aug 7
Learning Context-Free Grammars for Grammar-Constrained Decoding via Declarative Agentic Programming with Guarantees
arXiv:2608. 05493v1 Announce Type: cross Abstract: Language models (LMs) are increasingly used to interact with external services via programs written in domain-specific languages (DSLs).
arXiv AI
Sep 4
Grammar-Aligned Decoding
The paper introduces Grammar‑Aligned Decoding (GAD), addressing the issue that conventional grammar‑constrained decoding (GCD) can distort a large language model’s probability distribution, yielding grammatical but low‑likelihood outputs. GAD proposes an adaptive sampling method, Approximate Expected Futures (ASAp), which uses prior samples to over‑approximate future grammaticality, ensuring outputs remain both grammatical and faithful to the model’s conditional probabilities. Experiments on code generation and structured NLP tasks demonstrate that ASAp often produces higher‑likelihood outputs than existing GCD techniques while still enforcing the required grammatical constraints.