← Back to all news
arXiv Machine Learning By Zhiyun Shi

Rehearse Everything, Remember Nothing: Attic-KV Rehearses What Will Be Read

Read the original on arXiv Machine Learning →

The paper critiques the common practice of rehearing all key‑value cache entries before compression, showing that under tight budgets this strategy actually harms performance. Experiments on RULER and LongBench reveal that full rereading retains far fewer useful entries than methods that rehearse nothing. The authors propose Attic‑KV, a training‑free rehearsal that quizzes the model with context‑based question‑answer pairs, which outperforms existing methods and scales better as compression budgets shrink.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 15

Latent Compaction with Test-Time Training

Latent Compaction is a test‑time training method that learns a compact, document‑specific key–value cache for large‑context language models. It initializes cache entries using reconstruction importance, teacher attention, and key‑space diversity, then fits attention outputs and log‑normalizers to the full cache before jointly optimizing the entire cache via end‑to‑end distillation. Experiments on RULER, LongBench, and Phonebook Lookup show significant accuracy gains over budget‑matched compression baselines, achieving 96.7% accuracy at 32× compression on RULER and transferring well across model scales and families.

By Yingfa Chen, Xingyu Shen, Chaojun Xiao, Xu Han
efficiency
More like this →
arXiv AI
Oct 21

REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents

The paper introduces REM (Reusable Experience Memory), a framework that manages the lifecycle of experience for large language model agents performing long-horizon coding tasks. REM consolidates completed trajectories into compact, phase-structured documents, injects them at task start, and evicts them after a single exposure, making the memory system responsive to the agent’s needs. Experiments on repository-level bug fixing show REM outperforms state‑of‑the‑art memory systems in retrieval accuracy and task resolution while reducing cost.

By Xinyi Lisa Chen, Pedro Herrero-Vidal, Xiaohan Chen, Xiaojing Yu, Cris Liu, Bingqing Song, Bin Xu, Suleiman A. Khan, You-Lin Chen
llmsagentsbenchmarks
More like this →
arXiv AI
Sep 3

Language Models Can Control Their Own Attention

The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.

By Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santos
llmsefficiency
More like this →