← Back to all news
arXiv AI By Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia, Xinyu Liu, Alexander Heckett, Jiayuan Liu, Eden Hartman, Vojt\v{e}ch Kova\v{r}\'{i}k, Sven Neth, Maria-Florina Balcan, Tuomas Sandholm, Vincent Conitzer

Fundamental Impossibility Results in AI Safety: Trading Off between Desiderata

Read the original on arXiv AI →

The paper investigates whether safe AI systems can simultaneously satisfy multiple desiderata, such as harmlessness and helpfulness. By modeling the interaction between an AI, a defender, and an attacker as a three‑player game, the authors prove a fundamental tradeoff: advice that benefits the defender must remain public, limiting its safety guarantees. They also explore whether the AI can manipulate information to favor only the defender and discuss implications for safer AI design.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 15

Math for AI safety: an invitation for mathematicians

The article "Math for AI safety: an invitation for mathematicians" calls for new mathematical tools to ensure AI remains understandable, controllable, and cooperative. It outlines specific mathematical fields—logic, game theory, probability, algebra, representation theory, analysis, and geometry—each paired with an open problem tailored for mathematicians without AI safety background. The piece invites researchers to contribute to designing AI that is legible, steerable, and aligned with human values.

By Lionel Levine
safety
More like this →
OpenAI Blog
Jul 10, 2019

Why responsible AI development needs cooperation on safety

We’ve written a policy research paper identifying four strategies that can be used today to improve the likelihood of long-term industry cooperation on safety norms in AI: communicating risks and benefits, technical collaboration, increased transparency, and incentivizing standards. Our analysis shows that industry cooperation on safety will be instrumental in ensuring that AI systems are safe and beneficial, but competitive pressures could lead to a collective action problem, potentially causing AI companies to under-invest in safety.

safety
More like this →