arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.
arXiv:2610.33949v1 Announce Type: cross Abstract: Pipelines that require structured outputs from hosted language models often specify a Pydantic schema in the prompt, validate the response, and retry...
The paper investigates the so‑called ‘structure tax’ that is often attributed to constraining large language model outputs to structured formats like JSON or XML. By evaluating various models, datasets, and schema designs, the authors find that the accuracy loss depends more on how the schema is organized than on the presence of structure itself. Specifically, reasoning‑first field ordering can match or surpass free‑form accuracy, whereas answer‑first ordering leads to significant drops, especially in smaller models. The study also shows that well‑designed schemas improve confidence calibration and the separability of correct versus incorrect representations in transformer layers.
arXiv:2609.06993v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly generate Markdown that is consumed by renderers, agents, code extractors, and structured downstream pipelin...
arXiv:2609.21509v1 Announce Type: new Abstract: When language models reason in chain-of-thought or exchange free-text intermediates, they serialize structured information into natural language. How m...
SERIAL-7 is a Korean‑English benchmark that evaluates large language models on their ability to generate machine‑verifiable structured outputs across seven representations. It introduces an executable verification framework that separates semantic derivation (Derive), constrained artifact generation (Realize), and cross‑representation preservation (Preserve), using 177 validated artifacts and 43 atomic representation constraints. The benchmark shows that reliable structured generation requires explicit, executable verification of how artifacts are realized and preserved, as evidenced by varying compliance rates across ten models.
The paper introduces MeMoDecode, a decoding framework that enforces both concrete and abstract syntax as well as well‑formedness constraints for domain‑specific languages (DSLs) during large language model generation. By mapping generated text to partial metamodel instances and retrying from the last consistent snapshot, MeMoDecode guarantees consistency without sacrificing alignment metrics. Experiments on four DSLs and six open‑weight LLMs show a 23% average improvement over unconstrained decoding and an 8% improvement over grammar‑constrained decoding, with modest runtime overhead.
arXiv:2601.08626v4 Announce Type: replace Abstract: Large language models (LLMs) excel at semantic understanding, yet their ability to reconstruct internal structure from scrambled inputs remains und...
The paper investigates whether constrained decoding harms or helps large language model (LLM) performance. It introduces a taxonomy of three constraint types—format, validity, and reasoning—and shows that suitable constraints can preserve or even improve performance by eliminating invalid paths. Experiments across multiple model families reveal that perceived performance drops are mainly due to non-equivalent prompting and output formats, while validity constraints boost reliability and reasoning constraints can enhance accuracy when they guide beneficial reasoning trajectories.
The paper introduces Brittlebench, a framework and evaluation pipeline that measures how sensitive large language models (LLMs) are to prompt variations. By applying semantics‑preserving perturbations to popular benchmarks, the authors show that model performance can drop by up to 12% and that such perturbations can change model rankings in 63% of cases. The study finds that these input variations can explain up to half of the performance variance for a given model, underscoring the importance of robust evaluation methods.
arXiv:2607. 09999v1 Announce Type: cross Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.
arXiv:2610.35179v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for codebook-based content analysis in computational social science, where codebooks translate theor...