← Back to all news
arXiv AI By Junwei Yao, Yongxi Zhou, Zike Yang, Zhijun Wang, Jianan Liu, Yanxian Chen, Wenbo Ye

Evaluating Context Compaction on the Questions That Come Later

Read the original on arXiv AI →

The study examines how language‑model agents compact context for a current question and whether that compacted context remains useful for subsequent questions. Using four corpora that annotate evidence for multiple questions per text, the authors compare various compaction methods against random deletion at the same token budget. While sentence‑level compaction improves accuracy on the current question (e.g., gpt‑6‑luna boosts MultiRC accuracy by 11.4 points at 25% tokens), it does not outperform random deletion on later questions; word‑level deletion (LLMLingua‑2) shows better performance on later MultiRC questions and retains more evidence content words, though its advantage varies across datasets and evaluation settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

Hugging Face Trending Papers
Aug 12

The Sleeping Agent: What Gist-Based Context Compression Loses and Why

Gist-based context compression---summarising older conversation history into compact representations---is a common approach in long-horizon language model agents, yet its effect on different types of memory retrieval is poorly understood. We use Salience-Weighted Consolidation (SWC), a biologically-inspired compression framework motivated by sleep-based memory consolidation, as a diagnostic probe to study when gist compression helps and when it hurts.

llmsagentsnlpsafety
More like this →
arXiv AI
Oct 5

Sentence-Level Context Sensitivity as a Training-Free Detector of Unsupported Content, Evaluated Against Trained Verifiers

The paper proposes a training‑free detector that uses sentence‑level context sensitivity to identify unsupported content in retrieval‑augmented generation (RAG) answers. By re‑scoring each sentence with full context, no context, and each chunk removed, the method flags the chunk whose removal most reduces a sentence’s likelihood as the likely source. Evaluated on RAGTruth, TofuEval, and RAGBench, the detector outperforms answer‑level faithfulness scores, achieving AUCs up to 0.745 and matching per‑chunk fact‑checkers while using only a fraction of the compute required by large‑language‑model judges.

By Mohamed Aly Bouke
llmsragnlpbenchmarks
More like this →
arXiv Computation and Language
Oct 16

Prune the Summary, Not the History: Source-Judged Compaction for Long-Horizon Agents

Long‑horizon LLM agents use context compaction to replace an overflowing interaction history with a natural‑language summary. The paper shows that these summaries mix factual environment data with the compressor’s reasoning, making it hard for the agent to distinguish the two. The authors introduce Source‑Judged Compaction (SJC), a training‑free post‑processing step that scores each summary entry for a source in the environment or previous summary and removes unsupported entries, improving cross‑run reliability by 2.1%–8.9% on AppWorld and OfficeBench without increasing token usage.

By Guanghui Min, Chen Chen
llmsagents
More like this →