The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper introduces ARCBench, a configurable benchmark for measuring rule compliance in language‑model agents that compact long histories to fit context windows. Using ARCBench, the authors find that rule violations after compaction rarely stem from rule deletion; instead, agents often ignore the rule while it remains present in the compacted history. The study shows that the source of violations depends on how the rule is delivered and on the compaction strategy, with targeted compactor attacks significantly increasing violations, and that simply retaining a rule in the compacted summary does not guarantee compliance.
arXiv:2606. 02965v2 Announce Type: replace Abstract: As large language models gain tool access and are deployed as autonomous agents capable of editing records, executing transactions, and modifying infrastructure, we still evaluate them based on the sole metric of task completion.
arXiv:2606. 22528v2 Announce Type: replace Abstract: Modern LLM agents increasingly rely on context compaction, summarization, or eviction to keep long-running sessions within a token budget.
arXiv:2606. 02965v1 Announce Type: new Abstract: Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all.
arXiv:2607. 02599v1 Announce Type: cross Abstract: Tool-using LLM agents are usually evaluated by final-answer correctness or LLM judges.
arXiv:2608.22752v1 Announce Type: new Abstract: A safety rule and an episodic log compete for the same tokens in an AI agent's context. When the budget overflows, both are summarized at the same rate...