The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.
By Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu, Meet Udeshi, Sandeep K. Shukla, Prashanth Krishnamurthy, Farshad Khorrami, Muhammad Shafique, Ramesh Karri
arXiv:2606. 02644v1 Announce Type: cross Abstract: Agentic scaffolds have dramatically improved LLM performance on complex, long-horizon tasks, yielding both broad benefits and amplified risks in domains like cybersecurity.
By Eliot Krzysztof Jones, Mateusz Dziemian, Matt Fredrikson, J Zico Kolter
arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.
By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang
arXiv:2602. 07666v4 Announce Type: replace-cross Abstract: DARPA's AI Cyber Challenge (AIxCC, 2023--2025) is the largest competition to date for building fully autonomous cyber reasoning systems (CRSs) that leverage recent advances in AI -- particularly large language models (LLMs) -- to discover and remediate vulnerabilities in real-world open-source software.
By Cen Zhang, Younggi Park, Fabian Fleischer, Yu-Fu Fu, Jiho Kim, Dongkwan Kim, Youngjoon Kim, Qingxiao Xu, Andrew Chin, Ze Sheng, Hanqing Zhao, Michael Pelican, David J. Musliner, Jeff Huang, Jon Silliman, Mikel Mcdaniel, Jefferson Casavant, Isaac Goldthwaite, Nicholas Vidovich, Matthew Lehman, Taesoo Kim
arXiv:2607. 24780v1 Announce Type: new Abstract: Evaluating frontier LLMs is challenging: static benchmarks suffer from contamination and saturation -- leaving users unable to distinguish top models and developers blind to specific failure modes -- while human preference is subjective.
By Xingyu Chen, Rui Wang, Zhaopeng Tu, Liefeng Bo
The paper shows that fine‑tuning a large language model (LLM) with a debate framework—where a generator and a critic compete and a weaker LLM judge adjudicates—reduces reward hacking compared to standard reinforcement learning from AI feedback (RLAIF). In experiments on mathematics tasks, the debate approach keeps the judge’s performance stable, achieving a 45% higher peak validation accuracy than the RLAIF baseline and mitigating the rapid exploitation of judge errors. Additional findings indicate that weakening the judge speeds hacking unless countered by extra debate rounds, that debate can override misalignment prompts, and that word‑limit constraints on critiques help balance the game and prevent judge hacking.
whyItMatters":"The study demonstrates a practical method to curb reward hacking in RL‑based AI systems, addressing a key obstacle for safely scaling AI oversight."
By Zachary Kenton, Lili Janzer, Rory Greig, Tian Huey Teh, Kirill Tyshchuk, Jonah Brown-Cohen, Harri Edwards, Senthooran Rajamanoharan, Noah Y. Siegel, Natasha Jaques, Rohin Shah
arXiv:2608.23181v1 Announce Type: cross
Abstract: As large language models (LLMs) continue to advance in coding capabilities, their potential in cybersecurity has drawn increasing research attention,...
By Jian Yang, Haau-Sing Li, Shawn Guo, Zixi Zhao, Yibo Tan, Jiajun Wu, Aishan Liu, Xianglong Liu, Tianyu Zheng, Bryan Dai, Chengran Yang
arXiv:2606. 13385v1 Announce Type: cross Abstract: Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences.
By Zihao Wang, Yiming Li, Yutong Wu, Zheyu Liu, Kangjie Chen, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
arXiv:2607. 14309v1 Announce Type: new Abstract: The rapid development of Large Language Models (LLMs) and Artificial Intelligent (AI) powered autonomous agents has fundamentally changed the existing forms of software governance.
By Nutan Kumar Naik, Aditya Kumar Saroj, Vijay Prasad Poudel, Saurav Samantray, Abhishek Patel
arXiv:2607. 04613v1 Announce Type: new Abstract: Autonomous agents are moving from sandboxed text generators to operators of code, data, and physical infrastructure, and they increasingly learn while deployed.
By Xue Qin, Simin Luan, Cong Yang, Zhijun Li
arXiv:2606. 04075v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become a dominant post-training paradigm, enabling large language models (LLMs) to learn from rewards.
By Wei Liu, Xinyi Mou, Hanqi Yan, Zhongyu Wei, Yulan He
arXiv:2607. 20255v1 Announce Type: cross Abstract: LLM-driven autonomous agents are reshaping offensive security.
By Andreas Happe, J\"urgen Cito, Jasmin Wachter