The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The study investigates how post‑training quantization (8‑bit vs. 4‑bit) affects language‑model agents’ ability to recover from temporary tool failures. Using Llama‑3.1‑8B‑Instruct and Qwen2.5‑7B‑Instruct on twenty deterministic tool‑use tasks across five prompts, the authors find that recovery performance varies with prompt, evaluation target, and scoring policy, sometimes favoring 8‑bit, sometimes 4‑bit, and even reversing direction under different conditions. The results highlight that a single prompt, task set, or scoring rule cannot reliably determine the robustness of quantized agents. whyItMatters":"The findings emphasize that evaluation design choices critically influence conclusions about quantized‑agent robustness, underscoring the need for comprehensive, matched‑task assessments before deployment decisions are made."
arXiv:2607. 27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless.
arXiv:2609.14758v1 Announce Type: cross Abstract: Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply...
The study investigates why small language model agents tend to repeat a tool call that just failed. By recording the failed call and its error message in the transcript, the authors measure a negative corrective gain—agents are more likely to repeat the failed action, with a drop of about 1.03 nats per token. The problem is traced to the harness design rather than the model’s understanding of errors, and the authors show that replacing the verbatim call with a runtime-generated description of the failure can reduce this backfiring effect by 76%.
arXiv:2603.01209v3 Announce Type: replace Abstract: In CodeAct, language-model agents write Python that calls tools and use execution feedback to choose actions. Persistent runtimes preserve Python v...
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.