RiskChainBench is a new benchmark that pairs 3,600 synthetic token‑text restoration inputs with 600 human‑labeled local web environments to evaluate how well models can recover obfuscated platform messages and then investigate the associated websites. The benchmark measures both message restoration accuracy and the subsequent web‑investigation decision, using a fixed multimodal evidence judge to assess faithfulness, sufficiency, completeness, and consistency. Across ten models, performance varies widely, with entry recovery ranging from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%, highlighting execution failures as the main bottleneck.
By ZhuoXin Liu, Zhiming Ma, Ying Zhang, Mengzheng Yang, Yifan Wang, Zhengqi Huang, Yanhan Zhou, Zekun Lin, Jun Zhang, Shun Zhang, Yue Chen, Qiao Zhao, Peng Chen
arXiv:2609.06027v1 Announce Type: cross
Abstract: Search-augmented LLM agents are increasingly used for consumer decisions, making them vulnerable to Generative Engine Optimization (GEO) poisoning. E...
By Zhongan Bi, Qiwen Wang, Jianrong Jiang, Jigang Ding, Wenwen Xiong, Changhua Meng, Xuanang Gao, Kepeng Lin, Changjiang Jiang, Yiang Chen, Huan Yao, Wei Wang, Zhenyu Ma, Wenhui Dong
arXiv:2608. 04565v1 Announce Type: cross Abstract: LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking.
By Xuebin Li, Hanqing Zhao, Siyuan Liang, Kejiang Chen, Weiming Zhang, Dacheng Tao, Nenghai Yu
The paper introduces MIRAGE, a benchmark of 750 multi‑step decision tasks designed to test autonomous web agents’ investigative abilities across Wikipedia Forensics, Shopping Admin adjudication, and Reddit Moderation. Each task contains a misleading visible context and a hidden context that holds decisive evidence, allowing performance to be broken down into Investigation, Reasoning, and Decision Accuracy, with an added Investigative Hallucination Rate. Evaluation of eight LLM agents reveals three consistent patterns: agents often reach relevant pages but fail to extract decisive evidence, procedural hints improve investigation but not decision accuracy on Wikipedia tasks, and 12.6% of trajectories include fabricated facts.
By Syed Nazmus Sakib, Nafiul Haque, Tapodhir Karmakar Taton, Shahrear Bin Amin, Shifat E. Arman
arXiv:2606. 13385v1 Announce Type: cross Abstract: Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences.
By Zihao Wang, Yiming Li, Yutong Wu, Zheyu Liu, Kangjie Chen, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
arXiv:2603. 00801v2 Announce Type: replace Abstract: Language agents increasingly act as web-enabled systems that search, browse, and synthesize information from diverse sources.
By Shrey Shah, Levent Ozgur
arXiv:2605.29224v2 Announce Type: replace-cross
Abstract: AI agents augment large language models with external tools such as web retrieval, enabling grounded and up-to-date responses. However, incor...
By Aditya Nawal, Manit Baser, Mohan Gurusamy
arXiv:2606.17467v3 Announce Type: replace-cross
Abstract: Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and int...
By Aaditya Pai
The paper exposes a new vulnerability in Federated Retrieval-Augmented Generation (FedRAG) called Routing Hijacking, where a malicious client forges its semantic profile to attract target queries despite lacking relevant data. Experiments across three FedRAG routing architectures show that this attack consistently misroutes queries, causing downstream failures such as missing evidence, poisoning, incorrect answers, hallucinations, and sycophantic behavior. The authors propose a trust‑aware post‑routing framework that reweights clients based on evidence feedback, which effectively suppresses persistent hijacking and transfers to a learned neural router.
By Junjie Mu, Qiongxiu Li
TriShieldRAG introduces a three‑layered defense for Retrieval‑Augmented Generation: an Ingest Guard that screens documents, a Retrieval Scorer that re‑ranks based on trust, and a Cross‑LLM Consensus that validates evidence across three models. Against the original PoisonedRAG attack on the 2.68M‑passage Natural Questions corpus, the framework reduces attack success from about 79% to 1%. However, adaptive attacks that only alter document formatting can bypass the Ingest Guard and still achieve high success rates, revealing limits of layered defenses that rely on the same retrieved evidence.
By Susil Kumar Mohanty, Rohit Patel, Kosuru Yuvaraj, Jeenal Chaudhary, Disha Singhania
The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.
By Kimberly Milner, Minghao Shao, Nanda Rani, Haoran Xi, Venkata Sai Charan Putrevu, Meet Udeshi, Sandeep K. Shukla, Prashanth Krishnamurthy, Farshad Khorrami, Muhammad Shafique, Ramesh Karri
The paper introduces SEAV, a verification‑centric framework for evaluating jailbreak attempts against large language models. SEAV decomposes responses into ordered steps and checks both validity and correctness using LLM‑as‑a‑judge and retrieval‑grounded verification. The method reduces false positives by 14.9 percentage points on a strategic‑dishonesty diagnostic and reclassifies 22.1–51.0% of previously successful jailbreaks as invalid across multiple benchmarks.
By Qilong Wu, Sahil Wadhwa, Pranab Mohanty, Giri Iyengar, Varun Chandrasekaran