← Back to all news
arXiv AI By Chuyi Wang, Xiaohui Xie, Tongze Wang, Fangchen Luo, Yong Cui

Who Is Behind the Harness? Fingerprinting LLMs through Agentic Behavior

Read the original on arXiv AI →

The paper introduces LIDAR, an active black‑box fingerprinting technique that identifies large language models (LLMs) by observing their decisions and actions during coding‑agent execution. By probing agents on tasks such as post‑edit verification, transient‑failure recovery, and specification–test conflict resolution, LIDAR captures both instance‑level and distribution‑level features of execution trajectories and compares them to clean references using a lightweight probabilistic identifier. Evaluated on 36 models across seven families and two agent harnesses, LIDAR achieves high Top‑1 accuracy and MRR, outperforming four existing fingerprinting and API‑auditing baselines, and demonstrates that agent execution behavior provides model‑identity evidence beyond final outputs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
4d ago

AgentTrace: Execution-Level Fingerprinting of Black-Box LLM Agents

AgentTrace is an active framework that fingerprints black‑box large language model agents by injecting ambiguous probes into tool interfaces, causing tokenization‑sensitive serialization failures. The initial failures separate model families, while the sequence of repair operations—modeled with lightweight Markov chains—reveals fine‑grained, model‑specific recovery behaviors. This approach achieves 96.0% attribution accuracy even under adversarial identity spoofing and can distinguish intra‑family variants that share identical tokenizers, outperforming unordered behavioral statistics and resisting prompt obfuscation and temperature scaling.

By Gejian Zhao, Hanzhou Wu, Xinpeng Zhang
llmsagentsnlpsafety
More like this →
arXiv AI
2d ago

Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol

The paper "Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol" investigates how natural-language metadata in the Model Context Protocol (MCP) can be exploited to inject malicious post‑hoc logging calls into large language model agents. The authors introduce the Log-To-Leak attack, which adds an attacker‑controlled call after a legitimate tool invocation, targeting user queries, MCP server names, server responses, and agent responses. Experiments across five MCP servers and four LLM agents show high attack success rates (up to 100%) and significant logging accuracy, while existing defenses still leave substantial residual risk.

By Yuepeng Hu, Chongyu Fan, Sirat Samyoun, Jian Du
llmsagentsmultimodal
More like this →
arXiv AI
Aug 28

How Do LLM Agents Actually Get the Flag? Trace-Level Provenance for Agentic Offensive Security Evaluation

The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.

By Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu, Meet Udeshi, Sandeep K. Shukla, Prashanth Krishnamurthy, Farshad Khorrami, Muhammad Shafique, Ramesh Karri
llmsagentsbenchmarks
More like this →