The paper examines the challenge of safely allowing autonomous AI agents to improve themselves. It formalizes the problem by modeling agents as programs and defining improvement via a potentially uncomputable semantic score. The authors prove that in general, no terminating procedure can reliably accept only genuine improvements while never missing any, but they identify a practical bounded regime where exhaustive simulation can achieve these guarantees, noting its limitations.
Agentic AI systems can improve by searching longer, receiving additional support, or modifying how they propose and verify outputs. A performance score does not distinguish these mechanisms. We compar...
The paper introduces a formal framework for analyzing how agentic AI systems can improve themselves through longer searches, additional support, or changes in output verification. It defines bounded verification stages, alternating protocols, and terminal checkers, and shows how these mechanisms affect language classes and verification complexity. The authors prove that majority amplification preserves verification classes while existential acceptance can introduce errors, and they explore the implications for recursive self‑improvement and error budgeting.
arXiv:2605. 27381v3 Announce Type: replace-cross Abstract: Agents increasingly modify the procedures by which they solve tasks and improve themselves.
The paper proves that algorithmic safety verification for Turing‑complete, self‑modifying systems—whether fixed or recursively self‑improving—is fundamentally limited. Statistically, no verifier can be sound, complete, and tractable across unbounded domains, all finite configurations, or succinctly described finite environments, due to Rice’s, Gödel’s, Trakhtenbrot’s, coNP, and PSPACE barriers. Dynamically, even a single self‑modification step can render safety properties undecidable, and no total supervisory algorithm can guarantee correctness for all such transformations, though a monitor that raises alarms on violations remains feasible. whyItMatters":"The results show that formal safety guarantees for recursive self‑improvement are unattainable, highlighting intrinsic verification barriers for advanced AI systems."
arXiv:2609.08175v1 Announce Type: new Abstract: Harness self-evolution is the process by which an agent modifies its prompts, tools, code, or orchestration in response to task feedback while keeping...
arXiv:2607. 13070v1 Announce Type: cross Abstract: Safety claims on self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, or a README commitment.
SEVerA is a three‑stage framework that synthesizes self‑evolving agents with formal safety guarantees. It introduces Formally Guarded Generative Models (FGGM) that wrap parametric models in verified rejection samplers, ensuring every output satisfies a first‑order logic contract. The framework searches for candidate programs, verifies them against hard constraints, and then fine‑tunes them to improve task utility while preserving correctness, achieving zero constraint violations on tasks such as Dafny verification, symbolic math synthesis, and policy‑compliant agentic tool use.
arXiv:2609.06396v2 Announce Type: new Abstract: Recursive self-improvement (RSI) lets a system improve the model-building machinery from its own failures, so every later model inherits the gain. Yet...
arXiv:2610.15708v1 Announce Type: new Abstract: Coding agents based on frozen language models can improve their performance by refining the external context they rely on, without updating model weigh...
arXiv:2607. 13070v2 Announce Type: replace-cross Abstract: Safety claims for self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, a promise in a README.
arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...
arXiv:2603. 23129v3 Announce Type: replace Abstract: G\"odel agent realize recursive self-improvement: an agent inspects its own policy and traces and then modifies that policy in a tested loop.