The study investigates how post‑training quantization (8‑bit vs. 4‑bit) affects language‑model agents’ ability to recover from temporary tool failures. Using Llama‑3.1‑8B‑Instruct and Qwen2.5‑7B‑Instruct on twenty deterministic tool‑use tasks across five prompts, the authors find that recovery performance varies with prompt, evaluation target, and scoring policy, sometimes favoring 8‑bit, sometimes 4‑bit, and even reversing direction under different conditions. The results highlight that a single prompt, task set, or scoring rule cannot reliably determine the robustness of quantized agents.
whyItMatters":"The findings emphasize that evaluation design choices critically influence conclusions about quantized‑agent robustness, underscoring the need for comprehensive, matched‑task assessments before deployment decisions are made."
By Yuhe Hu
arXiv:2607. 14181v1 Announce Type: cross Abstract: The growing adoption of local inference frameworks such as Ollama has made it increasingly common for developers to run large code models on laptops and other resource-constrained hardware.
By Saima Afrin, Md. Zahidul Haque, Antonio Mastropaolo
Post-training quantization reduces the cost of deploying language-model agents, but its effect on recovery from temporary tool failures can depend on how recovery is evaluated. We compare 8-bit and 4-...
arXiv:2607. 20499v1 Announce Type: new Abstract: Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability.
By Sai Deekshith Lekkala, Jothi Prabha Appadurai, Rohith Reddy Bellibatlu, Manpreet Singh
The paper introduces CABRA, a synthetic benchmark that constructs coding tasks from scratch using call‑graph transformations and scales difficulty along four axes: function traversal, search, runtime resolution, and instruction following. Experiments with eight large language models and six coding agents on 6,840 CABRA tasks show that while LLM accuracy degrades with task size, agents maintain high performance by leveraging tools; larger tasks trigger more tool calls, and a separate study on SWE‑bench verifies that tool‑call counts better predict agent accuracy than lines of code edited. The authors argue that CABRA exposes weaknesses in LLMs that are hidden by tool use and highlights the importance of code‑understanding abilities beyond mere editing.
whyItMatters":"The study demonstrates that coding agents’ performance is more dependent on code‑understanding than on the amount of code edited, revealing a critical bottleneck for current coding agents."
By Nishant Balepur, Kiran Tomlinson, Tobias Schnabel
The paper introduces the QuantCode Model, a specialized language model for generating executable algorithmic trading code. It combines continued pretraining on trading framework code with supervised fine‑tuning on validated request‑to‑code pairs, achieving significant gains on the QuantCode‑Bench benchmark and improving both single‑turn and multi‑turn agentic success rates. The study also highlights the importance of retaining parsing and tool‑calling capabilities during domain specialization.
By Alexey Chernysh, Orkhan Ekhtibarov, Dmitry Zmitrovich