OpenAI Blog
Adversarial attacks on neural network policies
Read the original on OpenAI Blog →The Flow has not summarised this story yet — read it at OpenAI Blog.
Related stories
OpenAI Blog
Feb 24, 2017
Attacking machine learning with adversarial examples
Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to cause the model to make a mistake; they’re like optical illusions for machines. In this post we’ll show how adversarial examples work across different mediums, and will discuss why securing systems against them can be difficult.
arXiv Machine Learning
Jul 7
Training Verifiably Robust Agents Using Set-Based Reinforcement Learning
arXiv:2408. 09112v2 Announce Type: replace Abstract: Reinforcement learning policies parametrized by deep neural networks have achieved strong performance for continuous control, yet even small input perturbations may lead to unpredictable behavior.
arXiv AI
Jun 10
Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey
arXiv:2403. 00420v3 Announce Type: replace-cross Abstract: Deep Reinforcement Learning (DRL) is a subfield of machine learning for training autonomous agents that take sequential actions across complex environments.
arXiv Machine Learning
Sep 14
Robust Policy Optimization via Adversarial Importance Sampling
The paper introduces Adversarial Importance Sampling (Advis), a technique that leverages importance sampling over standard training trajectories to estimate and optimize worst‑case returns without extra environment interactions or auxiliary networks, thereby capturing long‑term robustness. It also presents advrl, a modular PyTorch library that consolidates existing robustness methods and adversarial attacks into single‑file implementations for easier prototyping and reproducible evaluation. Finally, the authors highlight that optimal adversarial hyperparameters do not transfer across agents, prompting evaluation against a broader set of attackers (6–14× more configurations) and demonstrate the effectiveness of their approach on continuous control tasks.
arXiv Machine Learning
Jun 19
Convex training of Lipschitz-regularized shallow neural networks
arXiv:2606. 19652v1 Announce Type: new Abstract: In this work, we introduce a training procedure for shallow neural networks that promotes robustness against adversarial attacks.
arXiv Machine Learning
Aug 27
Provable Privacy Attacks on Trained Shallow Neural Networks
The paper investigates provable privacy attacks on trained 2‑layer ReLU neural networks, specifically membership inference and data reconstruction. It demonstrates that the implicit bias of such networks can be leveraged to identify, with high probability, whether a given point was part of the training set in high‑dimensional, nearly orthogonal settings, and to construct a finite set containing a constant fraction of training points in a univariate setting. This work claims to be the first to reveal provable vulnerabilities arising from implicit bias in shallow neural networks.