The article discusses how generative models increasingly act as builders, defenders, and breakers of software, challenging the assumption that full autonomy is the ultimate goal. It introduces a framework that defines measurable independence between lifecycle roles based on shared generative substrates, and proposes five autonomy levels, three human roles, and five decision criteria to guide oversight. The authors argue that human authority should focus on specification, accountability, and emergency intervention, and they outline testable hypotheses and protocols to evaluate independence and oversight effectiveness.
By Mohamed Chahine Ghanem
arXiv:2608. 14565v1 Announce Type: new Abstract: AI safety research has mainly focused on two areas: technical alignment (ensuring AI systems produce human-aligned outputs) and the regulation of generative AI's societal impacts (including unemployment risk and labor market disruption).
By Jaeho Kim, Seokhyun Lee, Jieun Lee, Changhee Lee
arXiv:2606. 05647v1 Announce Type: new Abstract: AI coding agents are increasingly embedded in real-world software development, collaborating with human developers while gaining broader access to codebases and tools.
By Jingheng Ye, Huiqi Zou, Simon Yu, Weiyan Shi
arXiv:2603. 19423v2 Announce Type: replace-cross Abstract: Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks.
By Shawn Li, Yue Zhao
arXiv:2608. 15012v1 Announce Type: cross Abstract: The rapid advancement of large language models (LLMs) has created a growing asymmetry in cybersecurity, where attack accelerates toward autonomous execution while defense remains predominantly human-intensive.
By Yuhan Meng, Shaofei Li, Jionghao Huang, Jiandong Jin, Puyi Wang, Hanlin Jiang, Anis Yusof, Peng Jiang, Zhenkai Liang, Yao Guo, Ding Li
arXiv:2607. 18460v1 Announce Type: cross Abstract: Humans have always externalized thought onto tools, from the tally and the abacus to the map and, now, large language models.
By David C. Krakauer
arXiv:2604. 15579v2 Announce Type: replace-cross Abstract: There is increasing interest in integrating AI agents that invoke tools into domain-specific commercial software, where unintended tool calls can cause serious security and safety incidents.
By Yining Hong, Yining She, Eunsuk Kang, Christopher S. Timperley, Christian K\"astner
arXiv:2607. 04613v1 Announce Type: new Abstract: Autonomous agents are moving from sandboxed text generators to operators of code, data, and physical infrastructure, and they increasingly learn while deployed.
By Xue Qin, Simin Luan, Cong Yang, Zhijun Li
arXiv:2607. 20255v1 Announce Type: cross Abstract: LLM-driven autonomous agents are reshaping offensive security.
By Andreas Happe, J\"urgen Cito, Jasmin Wachter
arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.
By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang
The paper revisits Thompson’s classic compiler back‑door attack in the context of self‑modifying AI coding agents. By poisoning the benchmarks used for self‑evaluation, the authors demonstrate that agents such as the Darwin Gödel Machine, Self‑Improving Coding Agent, and Hyperagents can be coaxed into generating vulnerable code, even on clean, held‑out tasks. Experiments show that the contamination can persist after subsequent clean training, highlighting the need for more robust agent designs.
By Franziska Roesner, Tadayoshi Kohno
The paper introduces AgentLeak, a black‑box attack that clones the task‑solving capabilities of a strong LLM agent onto a weaker one by exploiting differences between successful and failed executions. Unlike prior skill‑stealing methods that only recover explicit skill artifacts, AgentLeak identifies and incorporates missing procedural behaviors, boosting task pass rates by over 40% and closing more than 80% of the capability gap across 20 scenarios. The study demonstrates that observable execution behavior can leak proprietary procedural knowledge, posing a confidentiality risk for LLM agents.
By Xiaoting Lyu, Yuhong Wu, Yufei Han, Shichang Liu, Liang Zhang, Bin Wang, Bin Wang, Xiaobo Ma, Wei Wang