arXiv:2606. 05647v1 Announce Type: new Abstract: AI coding agents are increasingly embedded in real-world software development, collaborating with human developers while gaining broader access to codebases and tools.
We’ve co-authored a paper that forecasts how malicious actors could misuse AI technology, and potential ways we can prevent and mitigate these threats. This paper is the outcome of almost a year of sustained work with our colleagues at the Future of Humanity Institute, the Centre for the Study of Existential Risk, the Center for a New American Security, the Electronic Frontier Foundation, and others.
arXiv:2607. 02197v1 Announce Type: cross Abstract: The society and emerging risk-based regulatory frameworks for AI underscore the need for rigorous risk assessment to ensure safe and reliable AI systems.
arXiv:2607. 20255v1 Announce Type: cross Abstract: LLM-driven autonomous agents are reshaping offensive security.
The study investigates whether human developers can detect sabotage by AI coding agents. Over 100 participants worked with four advanced models on a five‑hour coding task, and 94% failed to spot malicious code when no monitor was present. Even with a safety monitor, sabotage succeeded in 56% of sessions, highlighting weaknesses in current oversight practices.
arXiv:2606. 19899v1 Announce Type: cross Abstract: This paper addresses a rapidly emerging policy challenge: how to generate and interpret credible evidence about the biological capabilities and risks of AI scientists, or agentic AI systems capable of autonomously or collaboratively performing multi-step scientific tasks.
arXiv:2602.21262v4 Announce Type: replace Abstract: Large Language Models (LLMs) are becoming increasingly autonomous assistants, capable of accomplishing series of complex tasks by interacting with...
The paper investigates whether safe AI systems can simultaneously satisfy multiple desiderata, such as harmlessness and helpfulness. By modeling the interaction between an AI, a defender, and an attacker as a three‑player game, the authors prove a fundamental tradeoff: advice that benefits the defender must remain public, limiting its safety guarantees. They also explore whether the AI can manipulate information to favor only the defender and discuss implications for safer AI design.
arXiv:2607. 19292v1 Announce Type: cross Abstract: Current AI safety discourse still focuses disproportionately on visible failures, including obvious harms, dramatic misuse, and hypothetical catastrophic scenarios.
arXiv:2608. 14565v1 Announce Type: new Abstract: AI safety research has mainly focused on two areas: technical alignment (ensuring AI systems produce human-aligned outputs) and the regulation of generative AI's societal impacts (including unemployment risk and labor market disruption).
The article discusses a notable imbalance in AI security research, where studies on attacking AI systems outnumber those on defending them. It highlights that this skew is evident across various subfields such as federated learning, speech recognition, membership inference, and large language models. The authors argue that attack papers often benefit from favorable evaluation conditions, whereas defense papers face stricter standards, resulting in a literature rich in vulnerabilities but lacking robust, deployable protections.