arXiv:2606. 25987v1 Announce Type: cross Abstract: Large language models (LLMs) attain remarkable surface fluency on code, yet they neither formally guarantee the syntactic validity of their output nor leverage the hierarchical structure defining the target language.
arXiv:2608. 10137v1 Announce Type: cross Abstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step.
arXiv:2602.00612v3 Announce Type: replace Abstract: Diffusion Large Language Models (dLLMs) have demonstrated promising generative capabilities and are increasingly used to produce formal languages d...
arXiv:2602. 15983v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can translate natural language into optimization code, but silent failures pose a critical risk: code that executes and returns solver-feasible solutions may encode semantically incorrect formulations---a feasibility--correctness gap reaching 90 percentage points on compositional problems.
arXiv:2608. 06111v1 Announce Type: cross Abstract: Positional embeddings (PE) in Transformers encode token distance and order but are largely agnostic to \textit{syntactic structure}.
arXiv:2605. 15607v2 Announce Type: replace-cross Abstract: Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from pretraining remains poorly understood.
arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which a...
arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.
arXiv:2609.14384v1 Announce Type: new Abstract: What must a neural system be capable of to implement language? Current research annotates stimuli with linguistic variables and tests which electrodes,...
The paper investigates how transformers acquire deep semantic dependencies, proposing a mechanistic framework that frames learning as a competition between surface statistics and deep semantics. It identifies a "Gradient Starvation" effect that suppresses error signals for sparse semantic dependencies early in training, delaying structural reasoning until a sudden phase transition. The study also explains the success of Chain-of-Thought strategies and introduces a topology‑aligned contrastive objective that improves variable binding performance by more than twice the gain of standard fine‑tuning.
arXiv:2608. 05493v1 Announce Type: cross Abstract: Language models (LMs) are increasingly used to interact with external services via programs written in domain-specific languages (DSLs).
arXiv:2608.20359v1 Announce Type: new Abstract: Large language models (LLMs) are deployed for increasingly complex tasks involving planning and multi-step decision making, but high-quality performanc...
arXiv:2606. 18286v1 Announce Type: new Abstract: Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signal.