The paper investigates how token consumption varies across programming languages for coding agents. Five recent models were evaluated on Python, Java, Rust, and OCaml, revealing significant differences in token usage that persist across models. Analysis of agent trajectories shows that agents often generate non‑compiling solutions in unfamiliar languages, revise passing solutions, plan in code comments, distrust provided tests, and prototype in Python to sidestep unfamiliar languages.
arXiv:2606. 10933v1 Announce Type: new Abstract: LLM-based coding agents are usually evaluated in familiar software settings: mainstream languages, common libraries, and public repositories.
arXiv:2609.18959v1 Announce Type: new Abstract: LLM code-generation systems usually choose a target programming language before decoding and treat that choice as fixed. We show that, for language-fle...
arXiv:2603. 14501v2 Announce Type: replace-cross Abstract: Large Language Models excel in high-resource programming languages but struggle with low-resource ones.
arXiv:2607. 14816v1 Announce Type: cross Abstract: Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias.
Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text generation, its impact on code generation quality and programming conventions remains largely unexplored.
arXiv:2606. 03618v1 Announce Type: new Abstract: AI-assisted coding agents are bottlenecked by input-token cost.
arXiv:2610.22732v1 Announce Type: new Abstract: Advances in frontier coding agents have sharply increased the token costs of AI-assisted software engineering. Their token efficiency is typically eval...
The paper introduces PolyHuman, a dataset of human-written programs in C++, Java, and Python, to test whether large language models can judge functional equivalence across languages. Using this dataset, the authors evaluate several open-weight and proprietary LLMs, finding that models struggle more with harder problems, show language-specific biases, and rely partly on superficial similarity cues. They also observe run‑to‑run instability in GPT‑o4‑mini, concluding that current LLMs do not reliably capture functional equivalence within or across programming languages.
arXiv:2606. 14945v1 Announce Type: new Abstract: The autoresearch pattern enables autonomous experimentation by having a large language model (LLM) iteratively modify code to optimize a target metric.
arXiv:2607. 05188v1 Announce Type: new Abstract: A coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents about the program it is working on.
arXiv:2610.34633v1 Announce Type: cross Abstract: Repository-level code migration rewrites an entire project in a new programming language while preserving its observable behavior. It is challenging...
The paper investigates how large language models (LLMs) handle bug fixing versus problem solving in competitive programming. Using a dataset of ~3,000 Codeforces submissions and their human fixes, the authors compare LLM-generated patches to human patches and assess whether LLMs prefer to modify buggy code or generate new solutions. Results show that LLMs often alter more lines than necessary and sometimes produce entirely new solutions, performing better when allowed to generate solutions from scratch rather than patching existing code.