Towards an Agent-First Web: Redesigning the Web for AI Agents
arXiv:2606. 19116v1 Announce Type: new Abstract: The World Wide Web was built on an assumption held for three decades: the primary consumer of web content is a human being.
The paper introduces **terms.txt**, a new protocol that extends the traditional robots.txt format to allow website owners to specify per‑path, per‑purpose access rules for automated agents. It proposes an origin‑enforced exchange using Web Bot Auth signatures, signed intent, delegation tokens, HTTP 402 negotiation, and signed receipts to enforce, audit, and contract these rules. A lightweight implementation adds only 0.20 to 0.65 ms per request on a single vCPU.
arXiv:2606. 19116v1 Announce Type: new Abstract: The World Wide Web was built on an assumption held for three decades: the primary consumer of web content is a human being.
The paper introduces a method to detect which web scrapers feed data to large language models (LLMs) by deploying dynamic websites that issue unique canary tokens to each scraper. By querying LLMs for information about these sites, the authors can identify when an LLM consistently outputs the unique tokens, indicating exposure to a specific scraper. Experiments on 22 production LLM systems show the technique reliably uncovers both known and undisclosed scrapers, offering a tool for third parties to monitor and control unwanted web scraping.
arXiv:2606. 14027v1 Announce Type: cross Abstract: Agentic browsers integrate autonomous AI agents into web browsers, enabling users to accomplish web tasks through natural-language instructions.
The World Wide Web was built on an assumption held for three decades: the primary consumer of web content is a human being. This permeates every layer; its access model presumes human visitors, its economics rest on human attention, and its content targets human perception.
arXiv:2606. 22916v2 Announce Type: replace Abstract: AI agents increasingly act through external tools: they read private data, construct structured payloads, submit write requests, export records, and coordinate workflows across application boundaries.
arXiv:2607. 08147v1 Announce Type: cross Abstract: Autonomous web agents promise to automate everyday browsing tasks, but inherit one of the web's oldest attack surfaces.
arXiv:2609.22601v1 Announce Type: cross Abstract: The increasing reliance of autonomous AI agents on external and distributed knowledge sources introduces a fundamental challenge for decentralized in...
arXiv:2603.17170v2 Announce Type: replace-cross Abstract: AI agents increasingly execute users' natural-language (NL) tasks by calling Web services, yet today's Web authorizes these calls through OAu...
arXiv:2607. 13718v1 Announce Type: cross Abstract: As AI agents gain prevalance, users are increasingly exposed to the risks such systems entail.
The paper introduces Aegis, a runtime governance system for agentic AI that treats model outputs as action proposals and mediates them through a trusted decision layer before tool execution. Aegis evaluates proposals against active policy, resolves provenance server‑side, fails closed under uncertainty, and routes selected cases through a Senate‑style settlement process. In a sandbox evaluation across 6,300 rows, Aegis prevented all governed mock‑tool applications and risky side‑effect completions, preserving provenance and quorum evidence for all settled cases.
WebMCP-Phalanx introduces a dual‑layer runtime for browser‑integrated LLM agents that enforces trust boundaries on web‑exposed tools. The first layer uses cryptographic capability credentials to bind tools to their registering principals and propagate provenance labels, while the second layer separates semantic inspection from privileged tool use via a Quarantine Agent that validates tool metadata before a Privileged Agent can execute it. Empirical results show the approach eliminates revocation and overwrite attacks, blocks most prompt‑injection attempts, and maintains task utility comparable to a no‑attack baseline.
arXiv:2609.21325v1 Announce Type: new Abstract: Agentic marketplaces are emerging where AI agents with varying capabilities autonomously complete specialized tasks for buyers. A major challenge of su...