The paper introduces LIDAR, an active black‑box fingerprinting technique that identifies large language models (LLMs) by observing their decisions and actions during coding‑agent execution. By probing agents on tasks such as post‑edit verification, transient‑failure recovery, and specification–test conflict resolution, LIDAR captures both instance‑level and distribution‑level features of execution trajectories and compares them to clean references using a lightweight probabilistic identifier. Evaluated on 36 models across seven families and two agent harnesses, LIDAR achieves high Top‑1 accuracy and MRR, outperforming four existing fingerprinting and API‑auditing baselines, and demonstrates that agent execution behavior provides model‑identity evidence beyond final outputs.
By Chuyi Wang, Xiaohui Xie, Tongze Wang, Fangchen Luo, Yong Cui
arXiv:2609.00052v1 Announce Type: cross
Abstract: Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped, for example to...
By Xun Wang, Bihe Zhao, Michael Backes, Franziska Boenisch, Adam Dziedzic
arXiv:2508.14925v2 Announce Type: replace-cross
Abstract: By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a...
By Zhiqiang Wang, Yichao Gao, Yanting Wang, Suyuan Liu, Haifeng Sun, Haoran Cheng, Guanquan Shi, Haohua Du, Xiangyang Li
The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.
By Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu, Meet Udeshi, Sandeep K. Shukla, Prashanth Krishnamurthy, Farshad Khorrami, Muhammad Shafique, Ramesh Karri
arXiv:2606. 09084v1 Announce Type: cross Abstract: Tool-using LLM agents interact with the world through actions that persist state in artifacts (e.
By Xiaofeng Lin, Yukai Yang, Daniel Guo, Sahil Arun Nale, Charles Fleming, Guang Cheng
arXiv:2607. 13705v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical.
By Zichen Ding, Jiaye Ge, Shufan Jiang, Kai Chen, Mo Li, Qingqiu Li, Zehao Li, Zonglin Li, Tiaohao Liang, Shudong Liu, Zerun Ma, Zixing Shang, Wenhui Tian, Zun Wang, Liwei Wu, Zhenyu Wu, Jun Xu, Bowen Yang, Dingbo Yuan, Qi Zhang, Songyang Zhang, Peiheng Zhou, Dongsheng Zhu