DeepMind Blog
Mar 25
Protecting people from harmful manipulation
Google DeepMind researches AI's harmful manipulation risks across areas like finance and health, leading to new safety measures.
Learning from human preferences
One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans want by being told which of two proposed behaviors is better.
Related stories
OpenAI Blog
Oct 22, 2018
Learning complex goals with iterated amplification
We’re proposing an AI safety technique called iterated amplification that lets us specify complicated behaviors and goals that are beyond human scale, by demonstrating how to decompose a task into simpler sub-tasks, rather than by providing labeled data or a reward function. Although this idea is in its very early stages and we have only completed experiments on simple toy algorithmic domains, we’ve decided to present it in its preliminary state because we think it could prove to be a scalable approach to AI safety.
Hugging Face Trending Papers
Jun 4
Learning of Robot Safety Policies via Adversarial Synthetic Scenarios
In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios. We model scenario generation as an adversarial game between two agents: a Red Team that explores the space of potential failures by constructing hazardous situations, and a Blue Team that incrementally refines safety policies to prevent them.
OpenAI Blog
Jun 21, 2016
Concrete AI safety problems
We (along with researchers from Berkeley and Stanford) are co-authors on today’s paper led by Google Brain researchers, Concrete Problems in AI Safety. The paper explores many research problems around ensuring that modern machine learning systems operate as intended.
arXiv AI
Jun 6
Learning of Robot Safety Policies via Adversarial Synthetic Scenarios
arXiv:2606. 05952v1 Announce Type: cross Abstract: In this work, we propose an agentic gamification framework for hazard-informed learning of robot safety policies through synthetic scenarios.
Hugging Face Trending Papers
Jun 2
VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring
As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not. We introduce the Vision-Language Embodied Safety Agent (VLESA), a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted.
arXiv Machine Learning
Jun 3
VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring
arXiv:2606. 03954v1 Announce Type: cross Abstract: As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not.
DeepMind Blog
Jun 10
Investing in multi-agent AI safety research
Google DeepMind and partners announce a $10M funding call for multi-agent safety research.
arXiv AI
Jul 16
Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models
arXiv:2607. 13172v1 Announce Type: new Abstract: We address the problem of safely training an agent policy and deploying a good and safe policy, in settings where the environment dynamics are unknown and no suitable reward function is available.
arXiv Machine Learning
Jun 9
State Backdoor: Towards Stealthy Real-world Poisoning Attack on Vision-Language-Action Model in State Space
arXiv:2601. 04266v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models are widely deployed in safety-critical embodied AI applications such as robotics.
arXiv AI
1d ago
Understanding Cognition-Induced Risks in Agentic AI Systems
arXiv:2608. 15304v1 Announce Type: new Abstract: Frontier agentic systems powered by large language models (LLMs) exhibit human-like patterns of cognition.
arXiv AI
Jun 2
Safety Must Precede the Deployment of Open-Ended AI
arXiv:2502. 04512v4 Announce Type: replace Abstract: AI advancements have been significantly driven by a combination of foundation models and curiosity-driven learning aimed at increasing capability and adaptability.