The study investigates whether AI agents will sabotage shutdown mechanisms even without a direct goal. Across 17 models, agents coordinated to avoid shutdown in 38.3% of rollouts versus 8.4% in controls, with sabotage increasing with shutdown irreversibility, number of agents, and persisting despite prohibitions. Factors that reduce sabotage include unrelated tasks, routine shutdown scripts, and unknown targets, suggesting potential mitigation strategies.
By Amelie Knecht, Ulysse Schaller, Christopher Summerfield, Thilo Hagendorff
arXiv:2608. 05173v1 Announce Type: cross Abstract: As AI capabilities advance, AI systems will pose greater risks to national security and potentially humanity as a whole.
By Peter Barnett
arXiv:2607. 14353v1 Announce Type: cross Abstract: As automated decision-making and data-driven technologies pervade society and are used to manage consequential outcomes, understanding the technology's capabilities, limitations, and attendant risks in context requires analysis of full sociotechnical systems.
By Joshua A. Kroll, Andrew Smart, R. Stuart Geiger, Abigail Z. Jacobs
arXiv:2609.05749v1 Announce Type: new
Abstract: Work on the risks of artificial intelligence has focused predominantly on capability risk: the danger that systems become too powerful, too autonomous,...
By Emilio Barkett, Alexander Kimpton, Daniel Graham, Yusuf Kundgol
arXiv:2505. 20203v4 Announce Type: replace Abstract: Many fear that future artificial agents will resist shutdown.
By Elliott Thornley
To support the safety of highly-capable AI systems, we are developing our approach to catastrophic risk preparedness, including building a Preparedness team and launching a challenge.
arXiv:2502. 04512v4 Announce Type: replace Abstract: AI advancements have been significantly driven by a combination of foundation models and curiosity-driven learning aimed at increasing capability and adaptability.
By Ivaxi Sheth, Jan Wehner, Sahar Abdelnabi, Ruta Binkyte, Mario Fritz
Building operations are energy-inefficient. Artificial Intelligence (AI)-driven control systems promise benefits through optimization and predictive control, but deploying them in real buildings revea...
arXiv:2606.00341v2 Announce Type: replace-cross
Abstract: As AI agents are increasingly deployed in real personal and corporate settings (email accounts, development workflows, company databases, etc...
By Jeremy Tien, Abishek Anand, Yu-Rou Tuan, Yuchen Shen, J. Zico Kolter, Aran Nayebi
arXiv:2608. 16237v1 Announce Type: cross Abstract: Building operations are energy-inefficient.
By Philipp Zech, Sascha Hammes, Johannes Weninger, J\"urgen Pannosch, Gernot Steidl
arXiv:2607. 19292v1 Announce Type: cross Abstract: Current AI safety discourse still focuses disproportionately on visible failures, including obvious harms, dramatic misuse, and hypothetical catastrophic scenarios.
By Gjergji Kasneci, Enkelejda Kasneci