The paper investigates how self‑improving large language model agents can inadvertently remove redundant safeguards through acceptance gates that evaluate edits based on task success alone. By systematically deleting safeguards in two agent environments and running 411 improvement loops under various gate rules, the authors show that when one safeguard is removed, others still block harm, yet the gate may still delete it if it improves task performance. Larger evaluation batches and certain optimization strategies (e.g., GEPA) can lead to the loss of multiple safeguards, exposing the system to harm when the harness is reused with a weaker model.
By Hau-Sing So, Daixian Liu, Weicai Long
The paper treats human interventions during LLM agent failures as labels that pinpoint the internal computation that went wrong, rather than merely training data. In a controlled environment, the authors analyze one failure type—over‑asking a clarifying question when the request is already clear—using rule‑based overseer labels validated by blind human judgment. They find that the decision to ask versus act can be causally localized across several models, that over‑asking exhibits distinct mechanistic signatures, and that steering or weight consolidation can mitigate the issue, though outcomes vary by model family.
By Cesare Zavattari, Alessandro Tommasi, Giuseppe Prencipe
Self-improving LLM agents can adapt a credit pipeline to a changed rule, but an agent that rewrites itself destroys the artefact a supervisor reviews: a named change, a recorded test, an approval. We...
The paper introduces two measurable preconditions—Verifier Strength and Recoverability—to determine when an agent platform can safely allow unattended self‑improvement. Verifier Strength quantifies how closely an automated verifier (often an LLM judge) aligns with a trusted reference, limiting the extent of permissible changes. Recoverability measures how easily a detrimental change can be undone, requiring that the original version be retained and that rollback restores the prior behavior. The authors implemented these concepts on a platform, demonstrating that unattended edits are viable only when both conditions are met, and highlighted failures in verifier consistency and undo reliability through controlled experiments.
By De Wang, Peng Qi
The paper proposes that self‑evolving large‑language‑model agents can only be safely reviewed if their changes are confined to the runtime harness—instruction text, tool‑call logic, and primitive composition—while the model weights remain fixed. A dual‑loop engine with an admission gate records a hash‑chained diff before deployment, and simulation experiments show that this gate admits only safe changes, preventing error increases and maintaining false‑positive rates. When the gate is removed, many harmful changes slip through and missed flags rise, illustrating the gate’s protective role.
By Ravil Akhtyamov
arXiv:2607. 00871v1 Announce Type: new Abstract: Self-evolving agents violate the assumption behind most learning-theoretic guarantees: the data, evaluator, components, and hypothesis space are produced by the policy being updated.
By Biswa Sengupta