Constrained Decoding: Harmful or Beneficial? Revisiting Its Impact on LLM Performance

The paper investigates whether constrained decoding harms or helps large language model (LLM) performance. It introduces a taxonomy of three constraint types—format, validity, and reasoning—and shows that suitable constraints can preserve or even improve performance by eliminating invalid paths. Experiments across multiple model families reveal that perceived performance drops are mainly due to non-equivalent prompting and output formats, while validity constraints boost reliability and reasoning constraints can enhance accuracy when they guide beneficial reasoning trajectories.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• benchmarks