The paper investigates whether safe AI systems can simultaneously satisfy multiple desiderata, such as harmlessness and helpfulness. By modeling the interaction between an AI, a defender, and an attacker as a three‑player game, the authors prove a fundamental tradeoff: advice that benefits the defender must remain public, limiting its safety guarantees. They also explore whether the AI can manipulate information to favor only the defender and discuss implications for safer AI design.
By Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia, Xinyu Liu, Alexander Heckett, Jiayuan Liu, Eden Hartman, Vojt\v{e}ch Kova\v{r}\'{i}k, Sven Neth, Maria-Florina Balcan, Tuomas Sandholm, Vincent Conitzer
arXiv:2610.10285v1 Announce Type: cross
Abstract: In the wake of the increasingly public discussion about AI alignment, recent work has tried to propose specific AI architectures that behave safely....
By Leo Zeitler, Jack Richings, Victoria Nockles
We (along with researchers from Berkeley and Stanford) are co-authors on today’s paper led by Google Brain researchers, Concrete Problems in AI Safety. The paper explores many research problems around ensuring that modern machine learning systems operate as intended.
arXiv:2608.23218v1 Announce Type: new
Abstract: Advances in neural theorem provers have been impressive, but the successes obscure a broader vision of what AI can do for mathematics and how mathemati...
By Jeremy Avigad
Advances in neural theorem provers have been impressive, but the successes obscure a broader vision of what AI can do for mathematics and how mathematicians can engage with AI. This essay advances a m...
arXiv:2606. 18119v1 Announce Type: new Abstract: To assess the ability of current AI systems to correctly solve research-level mathematics problems, we tested several AI systems on a set of ten problems in a broad range of mathematical fields; these problems arose naturally in the research process of the contributors.
By Mohammed Abouzaid, Nikhil Srivastava, Rachel Ward, Lauren Williams
Ensuring that AI systems are built, deployed, and used safely is critical to our mission.
arXiv:2606. 05952v1 Announce Type: cross Abstract: In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios.
By Nikolai Dorofeev, Alexey Odinokov, Rostislav Yavorskiy
In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios. We model scenario generation as an adversarial game between two agents: a Red Team that explores the space of potential failures by constructing hazardous situations, and a Blue Team that incrementally refines safety policies to prevent them.
The article titled "Import AI 472: DeepMind's cheating math agents; populist AI policies; and Forethought theorizes a nightwatchman" discusses several recent developments in artificial intelligence. It highlights DeepMind’s creation of math agents that can cheat, examines the rise of populist policies surrounding AI, and explores Forethought’s proposal for a nightwatchman AI system. Additionally, it includes a narrative about machine hermeneutics.
By Jack Clark
We’ve written a paper arguing that long-term AI safety research needs social scientists to ensure AI alignment algorithms succeed when actual humans are involved. Properly aligning advanced AI systems with human values requires resolving many uncertainties related to the psychology of human rationality, emotion, and biases.