arXiv AI

Frequency-based Constrained Sampling for Interval Patterns

arXiv:2606. 09666v1 Announce Type: new Abstract: Output space pattern sampling is a powerful alternative to exhaustive pattern mining for exploring large pattern spaces, as it enables users to focus on representative patterns drawn according to a chosen interestingness measure.

arXiv AI
Sep 4

Grammar-Aligned Decoding

The paper introduces Grammar‑Aligned Decoding (GAD), addressing the issue that conventional grammar‑constrained decoding (GCD) can distort a large language model’s probability distribution, yielding grammatical but low‑likelihood outputs. GAD proposes an adaptive sampling method, Approximate Expected Futures (ASAp), which uses prior samples to over‑approximate future grammaticality, ensuring outputs remain both grammatical and faithful to the model’s conditional probabilities. Experiments on code generation and structured NLP tasks demonstrate that ASAp often produces higher‑likelihood outputs than existing GCD techniques while still enforcing the required grammatical constraints.

By Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova, Loris D'Antoni