Towards Data Science
Oct 13

Coding Agents Can Solve the Task — But How Much Work Did They Waste?

The article "Coding Agents Can Solve the Task — But How Much Work Did They Waste?" examines the performance of coding agents using real repositories and seeded agent decisions. It reports on 50,400 runs, highlighting metrics such as wasted turns, patch churn, and the effectiveness of different harness rules. The study provides insights into how these agents manage code modifications and the efficiency of their decision-making processes.

By Emmimal P Alexander
arXiv AI
Oct 5

Sentry: Learning to Recover from LLM Agent Failures at Test Time

Sentry is a failure‑management layer for large language model agents that learns from failures at test time. It retrieves relevant lessons from an external playbook when a failure occurs, verifies recovery without task rewards, and stores new lessons only if recovery succeeds, keeping the playbook out of the agent’s context. Across multiple benchmarks, Sentry outperforms both runtime‑intervention and context‑evolution baselines, and its lessons transfer to unseen tasks.

By Changxiu Ji, Amy Lu, Qizheng Zhang, Kunle Olukotun