When writing code, the same computation can be expressed in different programming languages and through different implementation strategies. We show that the activations of LLMs capture this shared st...
The paper introduces domain-specific languages (DSLs) as an explicit representation layer to bridge the gap between high-level natural-language abstractions and concrete actions. Experiments on the visually-grounded “Hexagons” task and the “Text2CAD” benchmark show that code generation using a human-curated DSL yields dramatic performance gains, especially at higher abstraction levels, while automatically induced DSLs only partially recover these gains. The study highlights that the right representation can close much of the performance gap caused by abstraction, but that automatically learning such representations remains a challenge.
The paper introduces Program Executability Prediction (PrEx), a task that asks large language models (LLMs) to determine whether a program is semantically valid or invalid and, if invalid, to identify the violated formal rule. To evaluate this, the authors create a dataset of systematically generated invalid programs derived from valid ones and test open‑source coding LLMs across different semantic formalisms, semantic shifts, and program splits (human‑written, LLM‑translated, fuzzer‑generated). Results show that LLMs rely more on pre‑training priors than on the provided semantics, performing poorly on modified semantics and with increasing program complexity.
arXiv:2607. 13921v1 Announce Type: cross Abstract: Languages with rich static semantics, such as Rust, provide stronger guarantees for AI-generated code, but their strictness makes generation more difficult.
arXiv:2602. 08857v2 Announce Type: replace-cross Abstract: Recent work has shown that the computations of Transformers can be simulated in the RASP family of programming languages.
arXiv:2606.12764v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to generate code at scale. Meanwhile, prior work has investigated whether training data may be r...
Large Language Models (LLMs) treat all context tokens uniformly, which creates prompt injection vulnerabilities and unreliable instruction execution. The paper introduces LLMON (LLM Object Notation), a post‑training framework that separates executable instructions from passive data and requires explicit identifier‑based execution. Using a 4.6 B‑token structured corpus and post‑training on Granite‑4.1 models, LLMON reduces indirect prompt‑injection success rates from 53–100 % to 0.02–5.6 % while maintaining general capabilities, and provides attention‑based interpretability metrics to diagnose failure modes.
arXiv:2505. 03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-trivial reasoning about program semantics.
arXiv:2610.15290v1 Announce Type: cross Abstract: We present Co-DSL, a framework in which users and LLMs co-design reusable spatial concepts for layout generation. In the first stage, our system allo...
arXiv:2606. 16496v1 Announce Type: cross Abstract: Large multimodal language models (LLMs) have emerged as powerful tools for guiding evolutionary search toward interpretable programmatic policies.
arXiv:2608.22638v1 Announce Type: cross Abstract: Copying a function from a chat window into an editor takes less than a second. For many uses of AI coding tools, that speed is the point; in settings...
The paper introduces "compile by training," a method that converts natural-language specifications into reusable neural functions. By generating task-specific examples with teacher models at compile time, a small adapter is trained for a compact interpreter, eliminating the need for remote model calls during execution. The approach achieves 83.6% semantic accuracy on FuzzyBench-Hard, though it incurs a higher compile-time cost of about a minute, and is demonstrated in applications such as a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English‑Claudish translator.