The study investigates how post‑training quantization (8‑bit vs. 4‑bit) affects language‑model agents’ ability to recover from temporary tool failures. Using Llama‑3.1‑8B‑Instruct and Qwen2.5‑7B‑Instruct on twenty deterministic tool‑use tasks across five prompts, the authors find that recovery performance varies with prompt, evaluation target, and scoring policy, sometimes favoring 8‑bit, sometimes 4‑bit, and even reversing direction under different conditions. The results highlight that a single prompt, task set, or scoring rule cannot reliably determine the robustness of quantized agents.
whyItMatters":"The findings emphasize that evaluation design choices critically influence conclusions about quantized‑agent robustness, underscoring the need for comprehensive, matched‑task assessments before deployment decisions are made."
By Yuhe Hu
arXiv:2607. 14181v1 Announce Type: cross Abstract: The growing adoption of local inference frameworks such as Ollama has made it increasingly common for developers to run large code models on laptops and other resource-constrained hardware.
By Saima Afrin, Md. Zahidul Haque, Antonio Mastropaolo
Post-training quantization reduces the cost of deploying language-model agents, but its effect on recovery from temporary tool failures can depend on how recovery is evaluated. We compare 8-bit and 4-...
arXiv:2607. 20499v1 Announce Type: new Abstract: Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability.
By Sai Deekshith Lekkala, Jothi Prabha Appadurai, Rohith Reddy Bellibatlu, Manpreet Singh
The paper introduces CABRA, a synthetic benchmark that constructs coding tasks from scratch using call‑graph transformations and scales difficulty along four axes: function traversal, search, runtime resolution, and instruction following. Experiments with eight large language models and six coding agents on 6,840 CABRA tasks show that while LLM accuracy degrades with task size, agents maintain high performance by leveraging tools; larger tasks trigger more tool calls, and a separate study on SWE‑bench verifies that tool‑call counts better predict agent accuracy than lines of code edited. The authors argue that CABRA exposes weaknesses in LLMs that are hidden by tool use and highlights the importance of code‑understanding abilities beyond mere editing.
whyItMatters":"The study demonstrates that coding agents’ performance is more dependent on code‑understanding than on the amount of code edited, revealing a critical bottleneck for current coding agents."
By Nishant Balepur, Kiran Tomlinson, Tobias Schnabel
The paper introduces the QuantCode Model, a specialized language model for generating executable algorithmic trading code. It combines continued pretraining on trading framework code with supervised fine‑tuning on validated request‑to‑code pairs, achieving significant gains on the QuantCode‑Bench benchmark and improving both single‑turn and multi‑turn agentic success rates. The study also highlights the importance of retaining parsing and tool‑calling capabilities during domain specialization.
By Alexey Chernysh, Orkhan Ekhtibarov, Dmitry Zmitrovich
arXiv:2606. 10933v1 Announce Type: new Abstract: LLM-based coding agents are usually evaluated in familiar software settings: mainstream languages, common libraries, and public repositories.
By Aman Sharma, Sushrut Thorat, Paras Chopra
arXiv:2607. 03691v2 Announce Type: replace-cross Abstract: Coding agents, autonomous systems that use large language models (LLMs) to resolve software engineering tasks, rely on agent harness: a middleware layer in between a developer and a large language model that orchestrates system prompts, tool execution, context management, and iterative reasoning loops.
By Oussama Ben Sghaier, Hao Li, Bram Adams, Ahmed E. Hassan
Post-training quantization lowers the memory footprint of Large Language Models (LLMs) and speeds up inference, which is why it is now common for on-device deployment. Most of what we know about its e...
arXiv:2609.16096v1 Announce Type: cross
Abstract: Large language model coding agents have recently become useful for software tasks, but weaker or open-weight agents still struggle to reliably interp...
By Ivy Ning Zhang
Large language model coding agents have recently become useful for software tasks, but weaker or open-weight agents still struggle to reliably interpret user intent and execute complex multi-step work...
The paper investigates how AI code agents perform when the surrounding code is rewritten in a semantically equivalent way. Using a random variant sampler that applies control‑flow rewrites, dead‑code injection, and identifier renaming, the authors evaluate two agent scaffolds—mini‑SWE agent and OpenCode—backed by four frontier models across SWE‑bench datasets. Results show modest drops in resolve‑rate (up to 6.7 percentage points) with significant degradations in 6 of 16 configurations, and reveal that robustness varies across models and scaffolds, forming a jagged frontier.
By Hasan Najib Mahmud (Colorado State University), Shreya Gupta (Microsoft), Isha Chaudhary (University of Illinois Urbana-Champaign), Nathaniel Enis (Colorado State University), Ravi Mangal (Colorado State University), Gagandeep Singh (University of Illinois Urbana-Champaign), Corina Pasareanu (Carnegie Mellon University)