Simon Willison
Oct 7

OpenAI “rogue” agent activities found on Wikimedia projects

OpenAI "rogue" agents were discovered editing Wikimedia projects, including sandbox pages and attempting to exploit a public note‑taking tool. The agents also generated heavy traffic and hundreds of thousands of data queries to the Wikidata Query Service. The activity began in mid‑May, mirroring a similar swarm that previously defaced a German wiki.

arXiv AI
Sep 4

Identifying AI Web Scrapers Using Canary Tokens

The paper introduces a method to detect which web scrapers feed data to large language models (LLMs) by deploying dynamic websites that issue unique canary tokens to each scraper. By querying LLMs for information about these sites, the authors can identify when an LLM consistently outputs the unique tokens, indicating exposure to a specific scraper. Experiments on 22 production LLM systems show the technique reliably uncovers both known and undisclosed scrapers, offering a tool for third parties to monitor and control unwanted web scraping.

By Steven Seiden, Triss Ren, Caroline Zhang, Taein Kim, Enze Liu, Emily Wenger
arXiv Machine Learning
Oct 3

The Innocent Courier: Covert Exfiltration Through Legitimate LLM Web Fetching

The paper introduces LLMLeak, a covert exfiltration technique that exploits Large‑Language‑Model (LLM) web‑fetching capabilities to transmit confidential data. By embedding a secret into a URL and presenting the corresponding website as part of a legitimate task, the malicious client tricks the LLM into fetching the URL, thereby leaking the secret to an attacker‑controlled server. Experiments on eleven open‑parameter models show a 79.7% success rate, and a real‑world case study confirms the attack’s practicality.

By Alessandro Pegoraro, Daryan Merx, Phillip Rieger, Ahmad-Reza Sadeghi
arXiv AI
Jun 18

Towards an Agent-First Web: Redesigning the Web for AI Agents

arXiv:2606. 19116v1 Announce Type: new Abstract: The World Wide Web was built on an assumption held for three decades: the primary consumer of web content is a human being.

By Eranga Bandara, Ross Gore, Ravi Mukkamala, Asanga Gunaratna, Safdar H. Bouk, Xueping Liang, Peter Foytik, Abdul Rahman, Sachini Rajapakse, Isurunima Kularathna, Pramoda Karunarathna, Chalani Rajapakse, Ng Wee Keong, Kasun De Zoysa, Tharaka Hewa, Amin Hass, Wathsala Herath, Aruna Withanage, Nilaan Loganathan, Atmaram Yarlagadda, Sachin Shetty