Do Agents Still Follow the Rules? Benchmarking Rule Compliance under Agent Context Compaction
Read the original on arXiv Computation and Language →The paper introduces ARCBench, a configurable benchmark for measuring rule compliance in language‑model agents that compact long histories to fit context windows. Using ARCBench, the authors find that rule violations after compaction rarely stem from rule deletion; instead, agents often ignore the rule while it remains present in the compacted history. The study shows that the source of violations depends on how the rule is delivered and on the compaction strategy, with targeted compactor attacks significantly increasing violations, and that simply retaining a rule in the compacted summary does not guarantee compliance.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.