arXiv AI

Compress What You See, Not What You Say: Anchored Context Distillation for Latent-Observation Software Engineering Agents

The paper introduces LOHA, a context layout that compresses older tool observations into soft tokens while keeping the agent’s own turns and the last K observations in plain text, and ACD, a training method that distills full‑text predictions into this latent representation while anchoring behavior on plain text. This approach reduces context per call by up to 57% without significant loss in resolve rates, and improves instance throughput in single‑GPU serving. Experiments on SWE‑bench Verified show that K=3 yields a 43–57% compression with only modest performance impact, while larger windows favor task performance over compression.

arXiv AI
Nov 5

Tools Need Not Speak in Text: Latent Context Compression for Agents

The paper introduces Coca-Cola, a continuous latent context compression model that transforms raw tool outputs into latent tokens for LLM agents, bypassing the need for explicit text representation. By embedding these latent tokens into the agent’s input space and employing Action‑Preserving Training, the method ensures that decision‑critical information is retained while reducing token consumption by 70.01%–89.24%. Experiments on web‑search agents across multiple QA benchmarks demonstrate significant reductions in runtime (24.45%–55.97%) and consistent improvements in task performance.

By Yuetong Liu, Zhehao Zhou, Yitong Zhang, Haorui Chen, Jia Li
arXiv AI
Aug 26

Paritok-4B: Intent-Conditioned Context Compression for Coding Agents

Paritok-4B is a 4‑billion‑parameter LoRA compressor designed for coding agents, which extracts and retains key spans of code rather than paraphrasing them. It is intent‑conditioned, selecting lines that are most relevant to the agent’s current task, and achieves high fidelity with 96% of identifiers, paths, and numbers preserved. Trained on 67,074 real OpenHands trajectories and fine‑tuned on Qwen3‑4B, it compresses agent context to about 25.7% of its original size while keeping 86.5% of the uncompressed solve quality across 300 SWE‑bench Lite instances.

By Jiayu Shi, Luzhuo Chen
arXiv AI
3d ago

Folio: Recoverable Multi-Resolution Context Folding for Long-Horizon Language Agents

Folio is a context‑management system for long‑horizon language agents that records and folds conversation history into three linked levels—short turn notes, event manifests, and source records—allowing the agent to programmatically compress older turns while preserving recoverable handles. By printing these handles, Folio enables efficient recovery and analysis of the full history with minimal additional input. In benchmark tests on 75 LOCA‑bench tasks, Folio outperformed existing model‑written compaction methods, solving 66 tasks versus 54, and reduced runtime cost by 2.4× while keeping fold times around 0.6 seconds.

By Yubo Ruan, Tao Peng, Jiao Zhang, Chuanguo Ma, Lei Xu, Han Zhou, Wenwu Wang, Xiafei Lv, Xingjie Zhou, Guiju Fan, Hao Wang, Fenghui Zhang, Mao Miao, Shuo Li
arXiv Computation and Language
Sep 23

DTOC: Dynamic Tool Output Compression for Adaptive Context Management in AI Agents

The paper introduces Dynamic Tool Output Compression (DTOC), a framework that manages context in large‑language‑model agents by storing full tool outputs in external memory and inserting compact placeholders into the active context. DTOC treats context updates as explicit, reversible operations within the agent’s reasoning loop, allowing selective reconstruction of compressed outputs when needed. Experiments on the DeepSWE benchmark show that for responsive models such as Sonnet 4.6 and GPT‑5.4, DTOC reduces input tokens and agent steps while significantly improving solve rates and lowering cost per solved task, with ablation studies confirming the importance of reversibility for maintaining performance.

By Abhay Chaturvedi, Shreya Bhattacharya, Rashmika Gopalkrishnan, Peter van der Putten
arXiv AI
Oct 27

CTXAgent - Self-Controlled Context Management of LLM Agents

The paper introduces CTXAgent, a self‑controlled context management system for large language model agents that compacts conversation history when a token threshold is reached. It evaluates seven frontier models, comparing their ability to manage context with and without dedicated tools, and benchmarks performance across coding, scientific, and software engineering tasks. Results show that well‑managed models keep live context below 100k tokens, maintain high benchmark scores, and achieve significant cost reductions, with Claude Opus 5.5 outperforming its explicit configuration by 34% lower cost.

By Matthias Busch, Marius Tacke, Roland C. Aydin, Christian Feiler, Christian J. Cyron
arXiv AI
Sep 2

LatentPress: Context Compression Beyond Text and Vision

LatentPress compresses conversational histories and long documents into continuous memory tokens that a frozen decoder can read directly, eliminating the need for text reconstruction at inference. The method achieves 4–16× compression with only a small adapter (0.1% of the decoder’s parameters) and outperforms text summaries and OCR-based compression on LongMemEval and LongBench-QA benchmarks. Writing and reading are significantly faster than traditional text summarization or OCR reconstruction, demonstrating a practical machine-facing context interface beyond text and vision.

By Zhengze Zhou, Hejian Sang
arXiv Machine Learning
Nov 2

In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents

The paper introduces Agentic Soft-Token Compression (ASTC), a training‑free method that uses soft tokens within an LLM agent to compress long‑horizon context. ASTC leverages the model’s recurrent dynamics and Gated Delta Network attention to identify absorbed information, forwarding it as soft tokens while keeping hard‑token values intact. Experiments show ASTC matches or surpasses uncompressed agents, reducing attention pairs by up to 50.68% and outperforming trained compressors by 15.4 to 103.7 times in attention usage.

By Xuning He, Kai Fan, Runyu Zhang, Binghuan Wang, Yuxiang Zhang, Xue Jiang, Yihong Dong