arXiv AI

Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks

arXiv:2606. 02875v1 Announce Type: new Abstract: Coding-agent benchmarks evaluate whether a single uninterrupted agent can resolve a repository issue.

arXiv AI
Aug 25

SWE Refactor Bench: Can Coding Agents Complete a Long-Horizon, Whole-Repository Stack Migration?

The paper introduces SWE Refactor Bench, a benchmark of 20 whole‑repository migrations aimed at testing whether coding agents can autonomously perform long‑horizon stack migrations. It employs a three‑stage evaluation—Migration Audit, Behavioural Tests, and Agentic Verification—to assess both migration completeness and behavioural correctness, revealing that only 5.4 % of runs succeed across all stages. The study shows that agents struggle with migration tasks, especially language rewrites, and highlights the need for more robust coding‑agent capabilities.

By Deyao Hong, Yizhe Chi, Wenyi Li, Xiaoqiu Wang, Mingju Gao, Kaisen Yang, Bingxiang He, Youjie Zheng, Calvin Xiao, Qinhuai Na
arXiv AI
5d ago

REVISE: Validity-Guided Recovery for Online Revisions in Agent Workflows

The paper introduces “Revise”, a runtime system that performs validity-guided, fine-grained recovery for online revisions in structured agent workflows. When a revision arrives, Revise intersects the change with recorded data and control dependencies, propagates the impact through the partially executed DAG, stops invalid work, preserves unaffected progress, and recomputes only the affected region. Experiments on real coding‑agent traces and LangGraph/LLMCompiler applications show that Revise matches a latest‑version oracle, reduces model calls by up to 56%, and improves service‑level objective goodput under load.

By Ruoling Qi, Xuaner Wu, Penghang Liu, Jian Chen, Yirui Liu
arXiv AI
Aug 26

When "Must" Becomes "Maybe": Constraint Weakening in LLM Agent Workflows

Large language model agents coordinate tasks via multi‑role, multi‑stage workflows that transform upstream state into intermediate artifacts such as summaries and plans. The study shows that when these artifacts are transformed—through compression, plan assimilation, or other handoff methods—the strict action‑binding constraints on upstream state can be weakened, turning mandatory requirements into optional information. In 1,296 synthetic episodes, direct handoff preserved all safety blockers, whereas transformed handoffs frequently deactivated or forbidden actions, but restoring full state fields or applying downstream verification can recover preservation.

By Yiheng Sun, Huifei Wang, Yancheng Zhu, Zhenyu Li, Zebin Zhao, Yifan Yuan
arXiv AI
Aug 26

The Handoff Tax: Continuing Non-Native Trajectories in LLM Agents

The paper investigates the cost–quality trade-offs involved when coding agents switch between low‑cost, low‑capability (LC) and high‑cost, high‑capability (HC) language models during long‑running tasks. By experimenting with different handoff directions, timings, and interfaces—full‑trajectory transfer, compaction, and trajectory removal—the authors find that full‑trajectory escalation recovers less than half of the LC‑to‑HC quality gap while adding significant cost, a penalty they call the handoff tax. Conversely, downshifting from HC to LC yields a more favorable cost‑quality balance, and the optimal interface depends on the direction of the handoff. whyItMatters":"The study quantifies how model handoffs impact both performance and expense, offering guidance for designing more efficient coding agents that balance cost and quality."

By Roy Ganz, Mor Shpigel Nacson, Adi Kalyanpur, Ron Litman