arXiv AI By Ruopei Sun, Jianfeng Cai, Kangwen Zhao, Wengang Zhou, Houqiang Li

Compact-Bench: Measuring What Context Compaction Costs Long-Horizon Coding Agents

Read the original on arXiv AI →

Compact‑Bench evaluates how context compaction affects long‑horizon coding agents by comparing full‑history and summary‑only continuations across 35 software‑engineering repairs, 146 agentic sessions, and 106 production chains. The benchmark tests 14 summarizer backbones and finds that stable damage ranges from 5‑64% of full‑history successes, with Claude Opus preserving the most repairs and DeepSeek‑Flash causing the most damage. It also shows that LLM summaries retain far fewer recurring details than mechanical extraction, and that failures can recur across all backbones and prompts, indicating that merely preserving information is insufficient for successful task continuation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 26

Benchmarks Are Surprisingly Compaction-Friendly

The paper investigates how context compaction—losing information to free up a language model’s context window—affects performance on long‑horizon agentic tasks. Using the OpenCode scaffold and five compaction strategies, the authors evaluate three frontier models on PaperBench, ProgramBench, and PostTrainBench, finding that compaction thresholds largely do not hurt performance for 1M‑context models, but can degrade GPT‑5.3‑Codex on certain benchmarks. The study also shows that compaction can silently drop critical task constraints, especially on ProgramBench and real user‑agent sessions from SWE‑chat, yet current benchmarks remain largely compaction‑friendly because objectives are fixed and progress is stored externally.

By Jeanne Salle, Hardik Bhatnagar, Matthias Bethge, Sahar Abdelnabi, Maksym Andriushchenko
arXiv Machine Learning
Sep 23

CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents

CliffCompaction is an autocompaction technique that reduces cost by up to 50% while maintaining or improving performance on benchmarks such as Terminal‑Bench and KernelBench. It achieves this by truncating or dropping content without rephrasing, ensuring compacted information remains faithful and preventing context drift. The method enables efficient test‑time scaling, matching or surpassing higher‑cost models like Opus 4.7 and GPT‑5.3 Codex, and delivers significant CUDA kernel speedups on KernelBench.

By Trang Nguyen, Eulrang Cho, Bingqing Chen, Tim Dettmers
arXiv Computation and Language
Oct 16

Prune the Summary, Not the History: Source-Judged Compaction for Long-Horizon Agents

Long‑horizon LLM agents use context compaction to replace an overflowing interaction history with a natural‑language summary. The paper shows that these summaries mix factual environment data with the compressor’s reasoning, making it hard for the agent to distinguish the two. The authors introduce Source‑Judged Compaction (SJC), a training‑free post‑processing step that scores each summary entry for a source in the environment or previous summary and removes unsupported entries, improving cross‑run reliability by 2.1%–8.9% on AppWorld and OfficeBench without increasing token usage.

By Guanghui Min, Chen Chen
arXiv AI
Oct 27

CTXAgent - Self-Controlled Context Management of LLM Agents

The paper introduces CTXAgent, a self‑controlled context management system for large language model agents that compacts conversation history when a token threshold is reached. It evaluates seven frontier models, comparing their ability to manage context with and without dedicated tools, and benchmarks performance across coding, scientific, and software engineering tasks. Results show that well‑managed models keep live context below 100k tokens, maintain high benchmark scores, and achieve significant cost reductions, with Claude Opus 5.5 outperforming its explicit configuration by 34% lower cost.

By Matthias Busch, Marius Tacke, Roland C. Aydin, Christian Feiler, Christian J. Cyron