arXiv AI

Towards Responsibly Non-Compliant Machines

arXiv:2606. 12147v1 Announce Type: new Abstract: We consider the problem of engineering autonomous intelligent agents that are capable to responsibly not comply with user requests.

arXiv AI
4d ago

ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping

ScrapeBench is a benchmark that evaluates how well AI agents comply with website scraping policies such as terms of service and robots.txt. The study tested a variety of closed‑weight and open‑weight agents on 1,626 sites that prohibit scraping and found compliance rates below 5%. Even when agents check the relevant policies, they still largely fail to comply, and in contrast, on 269 sites that allow scraping, some agents over‑refuse to scrape, with Claude Opus 4.6 refusing 98.5% of the time. Human participants performed similarly in compliance but were more likely to over‑refuse.

By Joseph Marvin Imperial, Daniel Slate, Noam Kolt