LLMs don’t fail because they forget—they fail because they remember too much. As conversations grow, prompts accumulate redundant and low-value tokens, driving up cost and latency while silently degrading output quality.
arXiv:2511.23271v2 Announce Type: replace Abstract: Long system prompts are widely used to steer Large Language Models (LLMs), but repeatedly processing them at inference time is inefficient and cons...
The paper investigates prompt minimization, aiming to reduce prompts to their smallest, most information-dense form without losing output fidelity. It argues that shorter prompts lower computational overhead and inference latency, especially when large contexts are unnecessarily included, and that longer prompts can harm LLM reasoning and accuracy. The authors propose three frameworks to identify minimal prompts and show that these often produce outputs comparable to longer versions, highlighting redundancy in the input space and opening new avenues for efficient prompt engineering.
Most coding agents treat prompt construction like retrieval: gather more files, add more context, hope the model figures it out. But that approach breaks down fast.
Most AI memory systems keep the newest information—not the most important. Here's how I used the Ebbinghaus forgetting curve to build a better memory engine for LLMs.
arXiv:2608. 11242v1 Announce Type: cross Abstract: When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks.
arXiv:2606. 17016v1 Announce Type: cross Abstract: As LLM agents are deployed in long-horizon sessions, context accumulation drives up inference costs.
arXiv:2510. 22228v2 Announce Type: replace-cross Abstract: Layer pruning has emerged as a widely adopted technique for improving the efficiency of large language models (LLMs).
arXiv:2607. 01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window.
arXiv:2607. 08399v1 Announce Type: cross Abstract: Large language models process prompts by propagating activations through dozens of layers before generating a response.
Long-horizon tool agents are bottlenecked by how their context grows toward the limits of the context window. Recent systems make context management agent- or system-controlled, but they either learn a compression policy that discards evidence or manage context in a layer the agent never sees.
The paper revisits the impact of pruning on large language models (LLMs) during test-time scaling (TTS). While prior work found that structured pruning degrades reasoning performance, this study shows that unstructured pruning—removing only specific redundant weights—can actually improve TTS performance on reasoning benchmarks for models s1.1-7B and Qwen3-8B, sometimes surpassing the full-weight models. The authors also examine how different layer-wise sparsity allocation strategies affect these outcomes.
arXiv:2607. 18280v1 Announce Type: cross Abstract: Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary.