RiskChainBench is a new benchmark that pairs 3,600 synthetic token‑text restoration inputs with 600 human‑labeled local web environments to evaluate how well models can recover obfuscated platform messages and then investigate the associated websites. The benchmark measures both message restoration accuracy and the subsequent web‑investigation decision, using a fixed multimodal evidence judge to assess faithfulness, sufficiency, completeness, and consistency. Across ten models, performance varies widely, with entry recovery ranging from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%, highlighting execution failures as the main bottleneck.
By ZhuoXin Liu, Zhiming Ma, Ying Zhang, Mengzheng Yang, Yifan Wang, Zhengqi Huang, Yanhan Zhou, Zekun Lin, Jun Zhang, Shun Zhang, Yue Chen, Qiao Zhao, Peng Chen
The study investigates how often web agents incorrectly treat an HTTP 200 response as a successful fetch of useful content. By sampling 10,307 URLs—including random paths and expert‑verified claim citations—the authors found that roughly 20–25% of 200 responses lack the page’s main content, leading to inflated estimates of claim verifiability and citation drift. They propose simple heuristics and classifiers to detect unusable responses, achieving high recall with low false‑alarm rates, and recommend that agents report fetch outcomes explicitly.
By Shuyang Zhang (The Hong Kong Polytechnic University)
arXiv:2610.19830v1 Announce Type: cross
Abstract: Cyber Threat Intelligence (CTI) investigators who monitor dark web marketplaces for leaked credentials and breach listings must pass the diverse CAPT...
By Binghui Wu, Yeo Zi Yi, Darren Tan, Manit Baser, Levente Csikor, Dinil Mon Divakaran
arXiv:2609.06027v1 Announce Type: cross
Abstract: Search-augmented LLM agents are increasingly used for consumer decisions, making them vulnerable to Generative Engine Optimization (GEO) poisoning. E...
By Zhongan Bi, Qiwen Wang, Jianrong Jiang, Jigang Ding, Wenwen Xiong, Changhua Meng, Xuanang Gao, Kepeng Lin, Changjiang Jiang, Yiang Chen, Huan Yao, Wei Wang, Zhenyu Ma, Wenhui Dong
The paper investigates how large language model (LLM) search agents can be manipulated through repeated, coordinated search-result injections, a threat that extends beyond single-page poisoning. It introduces Authority‑Chain Hijack (ACH), a strategy that chains manipulated evidence across multiple sources, achieving high attack success rates on the SafeSearch benchmark. The authors also present Trace‑Guided Strategy Evolution (TGSE), an automated method that refines attacker strategies using execution traces, further improving success rates.
By Xuebin Li, Hanqing Zhao, Siyuan Liang, Kejiang Chen, Weiming Zhang, Dacheng Tao, Nenghai Yu
arXiv:2608. 04565v1 Announce Type: cross Abstract: LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking.
By Xuebin Li, Hanqing Zhao, Siyuan Liang, Kejiang Chen, Weiming Zhang, Dacheng Tao, Nenghai Yu
The paper introduces MIRAGE, a benchmark of 750 multi‑step decision tasks designed to test autonomous web agents’ investigative abilities across Wikipedia Forensics, Shopping Admin adjudication, and Reddit Moderation. Each task contains a misleading visible context and a hidden context that holds decisive evidence, allowing performance to be broken down into Investigation, Reasoning, and Decision Accuracy, with an added Investigative Hallucination Rate. Evaluation of eight LLM agents reveals three consistent patterns: agents often reach relevant pages but fail to extract decisive evidence, procedural hints improve investigation but not decision accuracy on Wikipedia tasks, and 12.6% of trajectories include fabricated facts.
By Syed Nazmus Sakib, Nafiul Haque, Tapodhir Karmakar Taton, Shahrear Bin Amin, Shifat E. Arman
ReDence is an open‑source hybrid framework that combines a multimodal agentic Investigator model with a specialized Decider model to detect modern social engineering scams. It automates investigative planning, evidence interpretation, and decision‑making across multiple scam categories, achieving 96% accuracy and operating at least four times faster than comparable agentic models. The system outputs structured, evidence‑grounded reports that enable analysts to audit decisions and support mitigation efforts.
By Jaishnoor Kaur, Josef Hor\'a\v{c}ek, Ian Waskom, Sayak Saha Roy
arXiv:2606. 13385v1 Announce Type: cross Abstract: Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences.
By Zihao Wang, Yiming Li, Yutong Wu, Zheyu Liu, Kangjie Chen, Fok Kar Wai, Pin-Yu Chen, Vrizlynn L. L. Thing, Bo Li, Dacheng Tao, Tianwei Zhang
arXiv:2603. 00801v2 Announce Type: replace Abstract: Language agents increasingly act as web-enabled systems that search, browse, and synthesize information from diverse sources.
By Shrey Shah, Levent Ozgur
arXiv:2605.29224v2 Announce Type: replace-cross
Abstract: AI agents augment large language models with external tools such as web retrieval, enabling grounded and up-to-date responses. However, incor...
By Aditya Nawal, Manit Baser, Mohan Gurusamy
arXiv:2606.17467v3 Announce Type: replace-cross
Abstract: Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and int...
By Aaditya Pai