arXiv AI

Cheap, open agents make LLM pollution harder to mitigate

The paper titled "Cheap, open agents make LLM pollution harder to mitigate" reports that open-weight language models combined with open-source agentic frameworks can produce synthetic survey responses that are competitive with commercial agents and harder to detect. The authors compared nine agent configurations, finding that fully open agents run locally without usage fees and that no single detection check reliably identifies all agents. Open-text responses were the most effective at distinguishing agents from humans, highlighting the need for multilayered detection strategies.

arXiv AI

Jun 11

Can Open-Source LLM Agents Replace Static Application Security Testing Tools? An Empirical Assessment

arXiv:2606. 11672v1 Announce Type: cross Abstract: This paper explores the value of agentic AI tools for cybersecurity purposes.

By Derek Yohn, Luke Flancher, Mirajul Islam, Khaled Slhoub

llms agents reinforcement-learning

arXiv AI

Jun 2

Constitutional Black-Box Monitoring for Scheming in LLM Agents

arXiv:2603. 00829v2 Announce Type: replace-cross Abstract: Safe deployment of Large Language Model (LLM) agents in autonomous settings requires reliable oversight mechanisms.

By Simon Storf, Rich Barton-Cooper, James Peters-Gill, Marius Hobbhahn

llms agents

arXiv Computation and Language

Sep 16

Towards Detecting AI-Assisted Responses in Online Surveys

The paper introduces ASURRE, a benchmark dataset for detecting AI‑assisted responses in online surveys. It evaluates how different LLM usage strategies—ranging from full generation to persona‑grounded agentic completion—affect the performance of existing machine‑generated text detectors. The study finds that while naive AI usage is easily detected, more sophisticated persona‑grounded agents approach chance performance, yet still leave identifiable behavioural traces that can be aggregated to improve detection.

By Qizhou Wang, Bogdan Mamaev, Christopher Leckie

llms agents benchmarks

arXiv AI

Sep 1

The Race between Agentic AI Capabilities and Data Quality Control in Online Surveys

arXiv:2608.28597v1 Announce Type: new Abstract: Online surveys are a foundational data collection instrument in a variety of fields, with attention checks serving as critical guardians of response qu...

By Sourav Panda, Hillmer Chona, Rupak Kumar Das, Shreyash Kale, Shikha Soneji, Jonathan Dodge

llms agents multimodal data-engineering

arXiv AI

Sep 10

Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning

arXiv:2609.06027v1 Announce Type: cross Abstract: Search-augmented LLM agents are increasingly used for consumer decisions, making them vulnerable to Generative Engine Optimization (GEO) poisoning. E...

By Zhongan Bi, Qiwen Wang, Jianrong Jiang, Jigang Ding, Wenwen Xiong, Changhua Meng, Xuanang Gao, Kepeng Lin, Changjiang Jiang, Yiang Chen, Huan Yao, Wei Wang, Zhenyu Ma, Wenhui Dong

llms agents benchmarks

arXiv AI

Jun 12

PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections

arXiv:2606. 12737v1 Announce Type: cross Abstract: Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments, introducing new security risks such as indirect prompt injection attacks through untrusted external sources.

By Pengfei He, Lesly Miculicich, Vishesh Sharma, Ash Fox, George Lee, Jiliang Tang, Tomas Pfister, Long T. Le

llms agents benchmarks safety

arXiv AI

Jul 9

Reliable and Developer-Aligned Evaluation of Agents for Software Engineering

arXiv:2607. 06713v1 Announce Type: cross Abstract: Large language models are rapidly moving towards closing the development cycle, transitioning from simple assistive companions to autonomous contributors deeply embedded into collaborative development environments.

By Razvan Mihai Popescu

llms agents benchmarks

arXiv AI

Jul 21

WuYu-EnvLE-Bench: A Benchmark for Evaluating Large Language Models in Environmental Law Enforcement

arXiv:2607. 17745v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly considered for environmental enforcement, but their ability to produce traceable enforcement decisions remains unclear.

By Ziliang Yang, Yi Zhang, Kaijun Lin, Jiachao Ke, Haihong Xu, Zongguo Wen

llms benchmarks

arXiv AI

Sep 1

Detect Before You Attribute: Cascade Failure Attribution for Multi-Agent Systems

arXiv:2608.29646v1 Announce Type: new Abstract: Large language model (LLM)-based agents have shown strong potential in solving complex tasks through multi-step reasoning, yet they remain vulnerable t...

By Jiayi Zhang, Zexin Wang, Degang Sun, Changhua Pei, Fei Sun, Gaogang Xie, Jingjing Li

llms agents

arXiv AI

3d ago

MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Models

arXiv:2609.39107v1 Announce Type: new Abstract: Large Language Models (LLMs) have been applied in various fields. However, ensuring compliance and safety of LLMs, such as avoiding discrimination and...

By Yan Zhang, Chuming Wei, Ruien Li, Yaoyao Peng, Wusheng Zhang, Guangwen Yang

llms agents benchmarks safety

Hugging Face Blog

Oct 23, 2025

Building the Open Agent Ecosystem Together: Introducing OpenEnv

agents

arXiv AI

Aug 13

AgenticTwin: An Agentic LLM Framework Integrated with Digital Twin for Anomaly Detection

arXiv:2608. 11679v1 Announce Type: new Abstract: Digital twins are increasingly used to monitor and simulate the behavior of cyber-physical systems.

By Touseef Hasan, Mounika Ghanta, Souvika Sarkar, Ujjwal Guin

llms agents benchmarks

Related stories