arXiv AI

Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

arXiv:2607. 04281v1 Announce Type: cross Abstract: Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidence.

arXiv Machine Learning

Jun 19

Closing the Calibration Gap in Semantic Caching

arXiv:2606. 19719v1 Announce Type: cross Abstract: Semantic caching cuts LLM inference costs by serving a cached response to semantically similar queries.

By Aditeya Baral, Radoslav Ralev, Iliya Sotirov Zhechev, Srijith Rajamohan, Jen Agarwal

llms

arXiv AI

Jun 26

Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge

arXiv:2606. 26511v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) gives agents access to accumulated knowledge, but has no model of time.

By Neeraj Yadav

llms rag agents benchmarks

arXiv AI

Aug 11

KGCache: Amortized Subgraph Retrieval for KG Reasoning with LLMs

arXiv:2608. 07954v1 Announce Type: new Abstract: Large language models can answer knowledge-intensive questions more reliably when they are grounded with knowledge graphs, but systems such as Think-on-Graph and Reasoning-on-Graph repeatedly query the same graph neighborhoods across different questions.

By Uros Stanic, Changcheng Yuan, Sabuj Laskar, Ariful Azad

llms nlp

arXiv AI

Jun 3

Cost-Aware Query Routing in RAG: Empirical Analysis of Retrieval Depth Tradeoffs

arXiv:2606. 02581v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) faces a fundamental three-way tension: deeper retrieval improves factual grounding but inflates token costs and end-to-end latency.

By Sanjay Mishra

llms rag benchmarks safety

arXiv AI

Jul 20

Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching

arXiv:2607. 15516v1 Announce Type: cross Abstract: Production LLM deployments combine two cost-reduction primitives: prompt caching (a discounted rate for re-used token prefixes) and prompt compression (fewer tokens sent).

By Yan Song

llms rag benchmarks

arXiv AI

5d ago

Beyond Retrieval: Query-Conditioned Reuse of Long-Horizon Agent Trajectories

arXiv:2608. 12847v1 Announce Type: new Abstract: Retrieval can identify a past trajectory that may matter, yet it does not specify how an acting agent should use that trajectory after users, entities, constraints, or environment state have changed.

By Yifei Li, Heng Wang, Lingling Zhang, Muye Huang, Xinyu Zhang, Jiashuai Liu, Hang Yan, Rongman Xu

rag agents

arXiv AI

Jul 24

MiniCache: Reusable Program Caching with Small Model Interfaces for Efficient LLM Inference

arXiv:2607. 20507v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for program-aided reasoning, agentic decision making, and structured task execution, but these applications often incur high inference cost.

By Jingquan Chen, Jinghua Piao, Jie Feng, Shaogang Hu, Yong Li

llms agents efficiency

arXiv AI

Jun 6

QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

arXiv:2606. 05875v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost.

By Jianxin Yan, Wangze Ni, Zhenxin Li, Jiabao Jin, Zhitao Shen, Haoyang Li, Jia Zhu, Peng Cheng, Xuemin Lin, Lei Chen, Kui Ren

llms rag

Hugging Face Trending Papers

Aug 5

Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems

LLM agents that persist across sessions accumulate stored memories whose validity varies enormously by content type, yet existing memory architectures treat all memories as equally persistent and systematically contaminate retrieved context with outdated facts. We show that per-memory, type-conditioned temporal decay, a property of western scrub jay episodic memory, can be operationalized as an auto-classified coefficient $π_i$ in an external LLM-agent memory store, yielding ScrubJay-MEM: each memory is encoded as a jointly-bound What--Where--When tuple with an estimated perishability $π_i$ and utility horizon $τ_i$, retrieved by query-adaptive scoring, and revised retroactively at $O(1)$ LLM calls per update.

llms rag agents benchmarks

arXiv Machine Learning

Jun 30

Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation

arXiv:2606. 29947v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as rerankers in recommender systems, with the expectation that semantic understanding will help in cold-start and long-tail regimes.

By Zhe Dong (University of Maine at Presque Isle), Fang Qin (Stanford University), Manish Shah (Independent Researcher), Yicheng Wang (Independent Researcher)

llms rag benchmarks

arXiv AI

Jun 24

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

arXiv:2606. 24467v1 Announce Type: new Abstract: Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware.

By Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, Grace Li Zhang

llms efficiency

arXiv Machine Learning

Jul 20

Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluation

arXiv:2607. 16019v1 Announce Type: new Abstract: AI systems increasingly retrieve from records that revise themselves: issue threads, encyclopedic histories, policy logs, and long conversations.

By Zhaoyang Jiang, Zhizhong Fu, Zicheng Li, Yunsoo Kim, Jiacong Mi, Xuanqi Peng, Fei Teng, Honghan Wu

Related stories