AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv AI
Oct 7

Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

The paper introduces DEFER1, a deterministic-first enforcement system for large‑language‑model based multi‑agent systems that uses 28 checks to block most attacks and refers only a small fraction to human judges. In tests across four domains, DEFER1 reduces attack success from about 30% to roughly 3%, with 78% of attacks blocked deterministically and only a quarter reaching the judges. The study highlights that rules effectively handle clear policy violations while judges address ambiguous intent, but also reveals weaknesses such as a risk‑score gate that misclassifies many proposals.

By Shaswata Mitra, Raj Patel, Subash Neupane, Sudip Mittal, Md Rayhanur Rahman, Shahram Rahimi