Transduced language models (TLMs) combine a pretrained source language model with a finite‑state transducer to produce a language model over target strings. The paper introduces an unbiased stochastic estimator that resamples source prefixes without replacement and reweights them, allowing accurate estimation of target prefix probabilities while reducing computation compared to threshold‑pruned beam summing. Experiments on encyclopedic text, DNA, and DNA‑to‑amino‑acid transduction show improved compute–variance trade‑offs and significant runtime reductions, and the method also lowers estimated corpus surprisal in a reading‑time analysis without altering its conclusions.
By V\'esteinn Sn{\ae}bjarnarson, Samuel Kiegeland, Manuel de Prada Corral, Ryan Cotterell, Tim Vieira
arXiv:2602.00612v3 Announce Type: replace
Abstract: Diffusion Large Language Models (dLLMs) have demonstrated promising generative capabilities and are increasingly used to produce formal languages d...
By Yitong Zhang, Yongmin Li, Yuetong Liu, Jia Li, Xiaoran Jia, Zherui Li, Ge Li
arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.
By Avinash Reddy, Thayne T. Walker, James S. Ide, Amrit Singh Bedi
arXiv:2606. 00722v1 Announce Type: cross Abstract: Controlling language model outputs is essential for ensuring structural validity, reliability, and downstream usability, and diffusion language models are no exception.
By Hyundong Jin, Yo-Sub Han
The paper introduces Grammar‑Aligned Decoding (GAD), addressing the issue that conventional grammar‑constrained decoding (GCD) can distort a large language model’s probability distribution, yielding grammatical but low‑likelihood outputs. GAD proposes an adaptive sampling method, Approximate Expected Futures (ASAp), which uses prior samples to over‑approximate future grammaticality, ensuring outputs remain both grammatical and faithful to the model’s conditional probabilities. Experiments on code generation and structured NLP tasks demonstrate that ASAp often produces higher‑likelihood outputs than existing GCD techniques while still enforcing the required grammatical constraints.
By Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova, Loris D'Antoni
arXiv:2605. 18476v2 Announce Type: replace-cross Abstract: Coding and computation remain major bottlenecks in Markov chain Monte Carlo (MCMC) workflows, especially as modern sampling algorithms have become increasingly complex and existing probabilistic programming systems remain limited in model support, extensibility, and composability.
By Jungang Zou, Alex Ziyu Jiang, Qixuan Chen