arXiv Computation and Language

Signals and Queries: Token-Efficient Agentic Context Discovery

The paper introduces Signals and Queries (S&Q), a lightweight first-stage relevance filter for LLM agents that removes at least 30% of candidate policies before downstream evaluation while maintaining high recall. S&Q uses offline-generated activating examples and hard negatives to train a logistic classifier that predicts policy relevance via embeddings and vector dot products, eliminating per-turn LLM calls. Evaluated on 9,144 relevance labels across 254 conversation prefixes and 36 airline policies, S&Q outperforms top‑K and cosine‑threshold retrieval in precision at every recall level, consistently across two embedding providers.

arXiv Machine Learning
Jul 28

SMART: LLM-Augmented Hybrid Retrieval for Dynamic Product Ads

arXiv:2607. 23121v1 Announce Type: cross Abstract: Dynamic Product Ads (DPA) require retrieving relevant items from multi-million product catalogs, balancing two competing objectives: retargeting (re-surfacing known interests) and prospecting (discovering new categories).

By Congfei Zhang, Jingxiao Ma, Xiaodong Liu, Hsiang-wei Chao, Siman Wang, Ge Liu, Shantanu Aggarwal, Vincent Zhang, Meghana Missula, Rachel Liao, Zichu Li, Xiao Bai, Yunzhi Zhou, Yajun Wang, Zhe Liu, Jinchao Li, Yu Zhang
arXiv Computation and Language
Oct 12

Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence

The paper introduces a memory-augmented retrieval method that leverages token-level evidence instead of relying solely on single-vector embeddings. By performing sparse token matching and aggregating scores with whole-memory similarity, the approach improves retrieval performance across eight benchmarks and various model families without extra training. An optional low-rank query adapter can further boost results when supervised.

By Han Luo, Zhan Zhuang, Xu He, Jujia Zhao, Yu Li, Suqin Yuan, Dong Li, Lei Feng
arXiv AI
Jul 7

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference

arXiv:2605. 25475v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly expected to operate over long contexts, yet standard softmax attention incurs a KV cache that grows linearly with sequence length, quickly becoming the bottleneck for long context inference.

By Xintong Yang, Hao Gu, Binxing Xu, Lujun Li, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Yike Guo, Sirui Han