The paper introduces a method to detect which web scrapers feed data to large language models (LLMs) by deploying dynamic websites that issue unique canary tokens to each scraper. By querying LLMs for information about these sites, the authors can identify when an LLM consistently outputs the unique tokens, indicating exposure to a specific scraper. Experiments on 22 production LLM systems show the technique reliably uncovers both known and undisclosed scrapers, offering a tool for third parties to monitor and control unwanted web scraping.
By Steven Seiden, Triss Ren, Caroline Zhang, Taein Kim, Enze Liu, Emily Wenger
arXiv:2607. 10455v1 Announce Type: new Abstract: Autonomous CLI agents can now execute hundreds of actions across multi-hour sessions: writing code, executing shell commands, browsing the web, and managing cloud infrastructure, all with minimal human oversight.
By Kefan Song, Yanjun Qi
arXiv:2606.20910v2 Announce Type: replace-cross
Abstract: As AI web agents proliferate, combining large language models with autonomous, browser-level control, indiscriminate content scraping by web...
By Dayeon Kang, Hyejun Jeong, Jade Sheffey, Pubali Datta, Amir Houmansadr
Open source communities have been flooded with AI-generated contributions. In defense, they have written contribution rules to regulate coding agents' behavior, spanning from a total ban, mandatory disclosure, to verification gates and human sign-offs.
arXiv:2607. 26819v1 Announce Type: cross Abstract: Open source communities have been flooded with AI-generated contributions.
By Wenhao Yang, Runzhi He, Minghui Zhou
The paper introduces **terms.txt**, a new protocol that extends the traditional robots.txt format to allow website owners to specify per‑path, per‑purpose access rules for automated agents. It proposes an origin‑enforced exchange using Web Bot Auth signatures, signed intent, delegation tokens, HTTP 402 negotiation, and signed receipts to enforce, audit, and contract these rules. A lightweight implementation adds only 0.20 to 0.65 ms per request on a single vCPU.
By Rajarshi Chowdhury
arXiv:2606. 19116v1 Announce Type: new Abstract: The World Wide Web was built on an assumption held for three decades: the primary consumer of web content is a human being.
By Eranga Bandara, Ross Gore, Ravi Mukkamala, Asanga Gunaratna, Safdar H. Bouk, Xueping Liang, Peter Foytik, Abdul Rahman, Sachini Rajapakse, Isurunima Kularathna, Pramoda Karunarathna, Chalani Rajapakse, Ng Wee Keong, Kasun De Zoysa, Tharaka Hewa, Amin Hass, Wathsala Herath, Aruna Withanage, Nilaan Loganathan, Atmaram Yarlagadda, Sachin Shetty
arXiv:2608. 12323v1 Announce Type: cross Abstract: Specifying a penalty can paradoxically convert a legal obligation into a cost-benefit calculation that favors violation.
By Mika Okamoto, Ansel Kaplan Erol, Kutluhan Erol
The World Wide Web was built on an assumption held for three decades: the primary consumer of web content is a human being. This permeates every layer; its access model presumes human visitors, its economics rest on human attention, and its content targets human perception.
arXiv:2606. 02644v1 Announce Type: cross Abstract: Agentic scaffolds have dramatically improved LLM performance on complex, long-horizon tasks, yielding both broad benefits and amplified risks in domains like cybersecurity.
By Eliot Krzysztof Jones, Mateusz Dziemian, Matt Fredrikson, J Zico Kolter
arXiv:2610.35288v1 Announce Type: new
Abstract: CAPTCHAs and bot detection have helped protect Web services from automated abuse for decades by distinguishing humans from bots. However, the emergence...
By Zuyao Xu, Xiang Li
arXiv:2609.05587v1 Announce Type: new
Abstract: Existing evaluations of tool-using agents primarily measure whether an agent can successfully complete diverse tasks with tools. These evaluations gene...
By Hoyeol Yang, Woojung Song, Taewon Kim, Jonghyun Song, Seoyeon Park, Yohan Jo