The paper proves that algorithmic safety verification for Turing‑complete, self‑modifying systems—whether fixed or recursively self‑improving—is fundamentally limited. Statistically, no verifier can be sound, complete, and tractable across unbounded domains, all finite configurations, or succinctly described finite environments, due to Rice’s, Gödel’s, Trakhtenbrot’s, coNP, and PSPACE barriers. Dynamically, even a single self‑modification step can render safety properties undecidable, and no total supervisory algorithm can guarantee correctness for all such transformations, though a monitor that raises alarms on violations remains feasible.
whyItMatters":"The results show that formal safety guarantees for recursive self‑improvement are unattainable, highlighting intrinsic verification barriers for advanced AI systems."
By Jose Pascual Gumbau Mezquita
arXiv:2606. 28639v1 Announce Type: cross Abstract: This article establishes the foundational mathematical limits of Artificial General Intelligence (AGI) safety, proving that the core barrier is not the impossibility of an aligned state, but its structural unverifiability.
By Jose Pascual Gumbau Mezquita
arXiv:2609. 11326v2 Announce Type: replace-cross Abstract: We ask whether it can be certified algorithmically that a self-modifying computational system preserves a safety property at its next step (preservation) and along its whole evolution (persistence).
By Jose Pascual Gumbau Mezquita
arXiv:2606. 26057v1 Announce Type: cross Abstract: AI agents are granted access to tools, APIs, and other infrastructure, making them active principals in those systems.
By Seth Dobrin, {\L}ukasz Chmiel
arXiv:2609.06036v1 Announce Type: new
Abstract: Proposal-based controllers---learned policies, language-model planners, and other black-box \emph{generators}---are increasingly deployed behind runtim...
By Guangxi Wan, Yongbo Xie, Yuqi Liu, Qingwei Dong, Qingxin Li, Hongfei Bai, Peng Zeng
The paper introduces a semantic elevation operator that transforms static semantic questions about a program into dynamic questions about whether a property is preserved after the program rewrites itself. It proves that for intensional transformations the elevated property remains undecidable, showing that the class of non‑verifiable properties is closed under this operator and that repeated application climbs the arithmetical hierarchy to Υ02‑completeness. The work also demonstrates that no finite tower of verifiers can provide an unconditional certificate of preservation, and suggests a categorical perspective for future exploration.
By Jose Pascual Gumbau Mezquita
arXiv:2606. 26406v1 Announce Type: cross Abstract: We propose a complete architectural blueprint for safe artificial general intelligence based on a closed reentry loop (D I cycle).
By A. S. Ushakov, Yu. N. Berdinsk
arXiv:2607. 13070v1 Announce Type: cross Abstract: Safety claims on self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, or a README commitment.
By Deepak Soni
arXiv:2607. 22868v1 Announce Type: new Abstract: Runtime guardrails act before irreversible tool calls, but their guarantees depend on what policy state is representable, what a judge observes, and whether intervention changes future behavior.
By Shawn Ray
arXiv:2602. 20064v2 Announce Type: replace-cross Abstract: Large language models are increasingly deployed as agents: they plan, call tools, read untrusted data, and act on the results.
By Zac Garby, Andrew D. Gordon, David Sands
The paper introduces the concept of Evolutionary Safety for recursive self-improving AI, focusing on how safety properties evolve as an AI system and its successors change. It identifies key risks such as intent drift, error accumulation, and safety-property erosion, and presents a taxonomy covering agent state, model state, evaluation, environment, and update mechanisms. The authors propose methods for discovering and evaluating evolutionary risks, and outline governance principles for modification, selection, authorization, provenance, and recovery, while highlighting open problems for maintaining safety in persistent, adaptive, and recursively self-improving systems.
By Chang Gong, Jingping Bi, Di Yao, Xinjian Liang, Chao Xiang, Ruijie Guo
arXiv:2607. 13070v2 Announce Type: replace-cross Abstract: Safety claims for self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, a promise in a README.
By Deepak Soni