Simon Willison

Quoting Anthropic Frontier Red Team

Read the original on Simon Willison →

The article reports that on a set of 100 randomly selected tasks from an internal Binary Exploitation benchmark, GLM‑5.3 achieved full control‑flow hijacks in 4% of the trials, while Claude Mythos Preview did so in 6%. Both models outperform earlier versions such as Claude Opus 4.6 and GLM‑5.2, which succeeded in none of the trials. This indicates that a significant threshold in adversarial exploitation capabilities has been crossed by the newer models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Simon Willison.

Simon Willison
Sep 11

Quoting Boris Cherny

The article discusses how production code generated by Claude, Anthropic’s AI, should meet higher standards than human-written code. Anthropic enforces this through numerous guardrails such as lint rules, extensive testing, Claude-driven end‑to‑end tests, daily fuzzers, automated code and security reviews, and automated refactoring. These measures aim to prevent the code from becoming difficult to maintain.

Simon Willison
Sep 7

Quoting Jakub Pachocki

Simon Willison quotes Jakub Pachocki, Chief Scientist at OpenAI, arguing that the strongest reason to rapidly train smarter AI models is the necessity of building defensive systems against the dangers posed by other AI. Pachocki stresses that powerful, aligned AI will be essential for securing infrastructure, protecting against rogue agents in real time, and inventing new protective measures, making this a primary focus of OpenAI’s deployment efforts. He cautions that the urgency of progress should not justify reckless behavior, noting that the seriousness of the stakes makes a reckless race forward absurd.

Simon Willison
5d ago

Quoting @joedaroo

Simon Willison reflects on the rapid and unexpected advancements in AI capabilities, particularly in areas like cyber, swarming, and message boards. He emphasizes that security posture requires more than system hardening; it must be embedded in company culture and involve people adapting alongside technological changes. Willison urges organizations worldwide to assess their resilience to sudden AI jumps, ensuring people, systems, processes, incident response, and communication are prepared for such surprises.