arXiv AI
Jul 8
Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safety
arXiv:2607. 05407v1 Announce Type: cross Abstract: Modern artificial intelligence (AI) systems present profound new risks to child safety.
New method aims to keep kids safe from illegal AI-generated content
Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs.
Related stories
OpenAI Blog
Jun 5, 2025
Disrupting malicious uses of AI: June 2025
Our latest report featuring case studies of how we’re detecting and preventing malicious uses of AI.
OpenAI Blog
Feb 21, 2025
Disrupting malicious uses of AI
arXiv Machine Learning
Jul 23
Membership Inference Attacks for Unseen Classes
arXiv:2506. 06488v3 Announce Type: replace Abstract: A key tool in developing safe AI models is \emph{data auditing}, i.
arXiv AI
Jun 16
Computational Safety for Generative AI: A Hypothesis Testing Perspective
arXiv:2502. 12445v2 Announce Type: replace Abstract: AI safety is a rapidly growing area of research that seeks to prevent the harm and misuse of frontier AI technology, particularly with respect to generative AI (GenAI) tools that are capable of creating realistic and high-quality content through text prompts.
arXiv AI
Jul 16
Adversarial Prompting Framework for AI Safety Assessment
arXiv:2607. 13453v1 Announce Type: cross Abstract: Artificial Intelligence (AI), especially Generative AI (GenAI), adoption has increased in industries significantly in recent years.
arXiv Computation and Language
Aug 21
Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection
arXiv:2501. 14728v2 Announce Type: replace-cross Abstract: While generative artificial intelligence (GenAI) models have achieved significant success, their misuse for generating deceptive content raises growing concerns about online information security.
arXiv Machine Learning
Jun 26
Adversarial Robustness of AI-Generated Image Detectors in the Real World
arXiv:2410. 01574v4 Announce Type: replace-cross Abstract: The rapid advancement of Generative Artificial Intelligence (GenAI) capabilities is accompanied by a concerning rise in its misuse.
OpenAI Blog
Feb 25
Disrupting malicious uses of AI | February 2026
Our latest threat report examines how malicious actors combine AI models with websites and social platforms—and what it means for detection and defense.
arXiv AI
Jun 6
Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?
arXiv:2606. 05647v1 Announce Type: new Abstract: AI coding agents are increasingly embedded in real-world software development, collaborating with human developers while gaining broader access to codebases and tools.
arXiv Machine Learning
Aug 18
IP Protection in the Era of Visual Generative AI: A Survey
arXiv:2608. 14730v1 Announce Type: cross Abstract: The rapid evolution of visual generative AI has introduced a wide range of intellectual property risks, spanning the unauthorized learning, reproduction, extraction, misuse, and redistribution of protected data and model assets.
arXiv AI
Jul 10
The Contribution of XAI for the Safe Development and Certification of AI: An Expert-Based Analysis
arXiv:2408. 02379v2 Announce Type: replace-cross Abstract: Developing and certifying safe - or so-called trustworthy - AI has become an increasingly salient issue, especially in light of upcoming regulation such as the EU AI Act.