The paper proves that algorithmic safety verification for Turing‑complete, self‑modifying systems—whether fixed or recursively self‑improving—is fundamentally limited. Statistically, no verifier can be sound, complete, and tractable across unbounded domains, all finite configurations, or succinctly described finite environments, due to Rice’s, Gödel’s, Trakhtenbrot’s, coNP, and PSPACE barriers. Dynamically, even a single self‑modification step can render safety properties undecidable, and no total supervisory algorithm can guarantee correctness for all such transformations, though a monitor that raises alarms on violations remains feasible.
whyItMatters":"The results show that formal safety guarantees for recursive self‑improvement are unattainable, highlighting intrinsic verification barriers for advanced AI systems."
By Jose Pascual Gumbau Mezquita
The paper introduces a semantic elevation operator that transforms static semantic questions about a program into dynamic questions about whether a property is preserved after the program rewrites itself. It proves that for intensional transformations the elevated property remains undecidable, showing that the class of non‑verifiable properties is closed under this operator and that repeated application climbs the arithmetical hierarchy to Υ02‑completeness. The work also demonstrates that no finite tower of verifiers can provide an unconditional certificate of preservation, and suggests a categorical perspective for future exploration.
By Jose Pascual Gumbau Mezquita
arXiv:2606. 28639v2 Announce Type: replace-cross Abstract: We establish the mathematical limits of AGI safety in two forms: verifying a fixed system, and verifying that a certified safety property persists once the system self-modifies.
By Jose Pascual Gumbau Mezquita
arXiv:2303. 04203v4 Announce Type: replace Abstract: The theory of computation was built to answer Turing's question: what is effectively calculable by an unbounded, immortal, disembodied agent following rules?
By Xin Li
arXiv:2607. 13070v2 Announce Type: replace-cross Abstract: Safety claims for self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, a promise in a README.
By Deepak Soni
arXiv:2609.06036v1 Announce Type: new
Abstract: Proposal-based controllers---learned policies, language-model planners, and other black-box \emph{generators}---are increasingly deployed behind runtim...
By Guangxi Wan, Yongbo Xie, Yuqi Liu, Qingwei Dong, Qingxin Li, Hongfei Bai, Peng Zeng
arXiv:2607. 15629v1 Announce Type: cross Abstract: Topos causal models recast causal inference inside a topos: a causal world is a presheaf, an intervention is a characteristic map into the subobject classifier, and reasoning is carried out in the intuitionistic internal language.
By Karen Sargsyan
arXiv:2606. 15712v1 Announce Type: cross Abstract: We ask a structural question: given unreliable elementary problem-solvers, what organizations of them solve hard problems reliably, and what are the limits?
By Hidayet Aksu
arXiv:2606. 30788v1 Announce Type: new Abstract: Language models are often adapted in stages: a public skill phase, a private memory phase, and a later safety phase that learns to refuse outputs tied to the remembered entities.
By John Sweeney
arXiv:2607. 29496v1 Announce Type: new Abstract: We study transcript management for fixed, finite-precision causal Transformers.
By Sergey Salishev
arXiv:2607. 17469v1 Announce Type: cross Abstract: A randomized algorithm may terminate almost surely even though exceptional random tapes make it run forever.
By Yunbei Xu
arXiv:2607. 13070v1 Announce Type: cross Abstract: Safety claims on self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, or a README commitment.
By Deepak Soni