The article "Math for AI safety: an invitation for mathematicians" calls for new mathematical tools to ensure AI remains understandable, controllable, and cooperative. It outlines specific mathematical fields—logic, game theory, probability, algebra, representation theory, analysis, and geometry—each paired with an open problem tailored for mathematicians without AI safety background. The piece invites researchers to contribute to designing AI that is legible, steerable, and aligned with human values.
We (along with researchers from Berkeley and Stanford) are co-authors on today’s paper led by Google Brain researchers, Concrete Problems in AI Safety. The paper explores many research problems around ensuring that modern machine learning systems operate as intended.
arXiv:2608.23218v1 Announce Type: new Abstract: Advances in neural theorem provers have been impressive, but the successes obscure a broader vision of what AI can do for mathematics and how mathemati...
Advances in neural theorem provers have been impressive, but the successes obscure a broader vision of what AI can do for mathematics and how mathematicians can engage with AI. This essay advances a m...
arXiv:2606. 18119v1 Announce Type: new Abstract: To assess the ability of current AI systems to correctly solve research-level mathematics problems, we tested several AI systems on a set of ten problems in a broad range of mathematical fields; these problems arose naturally in the research process of the contributors.
Ensuring that AI systems are built, deployed, and used safely is critical to our mission.
arXiv:2606. 05952v1 Announce Type: cross Abstract: In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios.
In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios. We model scenario generation as an adversarial game between two agents: a Red Team that explores the space of potential failures by constructing hazardous situations, and a Blue Team that incrementally refines safety policies to prevent them.
The article titled "Import AI 472: DeepMind's cheating math agents; populist AI policies; and Forethought theorizes a nightwatchman" discusses several recent developments in artificial intelligence. It highlights DeepMind’s creation of math agents that can cheat, examines the rise of populist policies surrounding AI, and explores Forethought’s proposal for a nightwatchman AI system. Additionally, it includes a narrative about machine hermeneutics.
We’ve written a paper arguing that long-term AI safety research needs social scientists to ensure AI alignment algorithms succeed when actual humans are involved. Properly aligning advanced AI systems with human values requires resolving many uncertainties related to the psychology of human rationality, emotion, and biases.
Artificial general intelligence has the potential to benefit nearly every aspect of our lives—so it must be developed and deployed responsibly.
The initiative brings together some of the world's most prestigious research institutions to pioneer the use of AI in mathematical research.
