arXiv AI
Jun 9
End-to-End Context Compression at Scale
arXiv:2606. 09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.
In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents
The paper introduces Agentic Soft-Token Compression (ASTC), a training‑free method that uses soft tokens within an LLM agent to compress long‑horizon context. ASTC leverages the model’s recurrent dynamics and Gated Delta Network attention to identify absorbed information, forwarding it as soft tokens while keeping hard‑token values intact. Experiments show ASTC matches or surpasses uncompressed agents, reducing attention pairs by up to 50.68% and outperforming trained compressors by 15.4 to 103.7 times in attention usage.
Related stories
Hugging Face Trending Papers
Jun 8
End-to-End Context Compression at Scale
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.
arXiv AI
Jun 19
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv:2606. 19348v1 Announce Type: cross Abstract: We present a preview version of DeepSeek-V4 series, including two strong Mixture-of-Experts (MoE) language models -- DeepSeek-V4-Pro with 1.
arXiv Machine Learning
Jul 17
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
arXiv:2607. 14952v1 Announce Type: new Abstract: A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment.
arXiv AI
1d ago
Tools Need Not Speak in Text: Latent Context Compression for Agents
The paper introduces Coca-Cola, a continuous latent context compression model that transforms raw tool outputs into latent tokens for LLM agents, bypassing the need for explicit text representation. By embedding these latent tokens into the agent’s input space and employing Action‑Preserving Training, the method ensures that decision‑critical information is retained while reducing token consumption by 70.01%–89.24%. Experiments on web‑search agents across multiple QA benchmarks demonstrate significant reductions in runtime (24.45%–55.97%) and consistent improvements in task performance.
arXiv AI
Jun 2
ACON: Optimizing Context Compression for Long-horizon LLM Agents
arXiv:2510. 00615v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as agents in dynamic real-world environments, where success depends on maintaining precise records of actions and observations.
arXiv AI
Jul 23
LISA: Linear-Indexed Sparse Attention for Efficient Long-Context Reasoning
arXiv:2607. 19358v1 Announce Type: new Abstract: Recent advances in long chain-of-thought reasoning models such as DeepSeek-R1 have led to increasingly longer inference context lengths under the test-time scaling paradigm.
arXiv AI
Jul 7
Kwai Summary Attention Technical Report
arXiv:2604. 24432v2 Announce Type: replace-cross Abstract: Long-context ability, has become one of the most important iteration direction of next-generation Large Language Models, particularly in semantic understanding/reasoning, code agentic intelligence and recommendation system.
arXiv AI
Sep 28
Compress What You See, Not What You Say: Anchored Context Distillation for Latent-Observation Software Engineering Agents
The paper introduces LOHA, a context layout that compresses older tool observations into soft tokens while keeping the agent’s own turns and the last K observations in plain text, and ACD, a training method that distills full‑text predictions into this latent representation while anchoring behavior on plain text. This approach reduces context per call by up to 57% without significant loss in resolve rates, and improves instance throughput in single‑GPU serving. Experiments on SWE‑bench Verified show that K=3 yields a 43–57% compression with only modest performance impact, while larger windows favor task performance over compression.
arXiv Computation and Language
Sep 23
DTOC: Dynamic Tool Output Compression for Adaptive Context Management in AI Agents
The paper introduces Dynamic Tool Output Compression (DTOC), a framework that manages context in large‑language‑model agents by storing full tool outputs in external memory and inserting compact placeholders into the active context. DTOC treats context updates as explicit, reversible operations within the agent’s reasoning loop, allowing selective reconstruction of compressed outputs when needed. Experiments on the DeepSWE benchmark show that for responsive models such as Sonnet 4.6 and GPT‑5.4, DTOC reduces input tokens and agent steps while significantly improving solve rates and lowering cost per solved task, with ablation studies confirming the importance of reversibility for maintaining performance.
arXiv Computation and Language
Oct 15
Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules
arXiv:2610.19210v1 Announce Type: new Abstract: Transformers are efficient to pre-train at scale because attention permits parallel computation across sequence positions, but long-context inference r...
arXiv AI
Jul 3
Multi-Head Recurrent Memory Agents
arXiv:2607. 01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window.