arXiv AI

Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents

arXiv:2606. 08151v1 Announce Type: new Abstract: Tool-using LLM agents often fail not because relevant text is absent, but because decisive evidence is not selected, compressed, or surfaced at action time.

arXiv AI

Jul 29

Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents

arXiv:2607. 24882v1 Announce Type: cross Abstract: Modern coding agents are usually evaluated by whether they eventually produce a correct patch, but patch generation depends on an earlier context-acquisition stage: finding the repository files needed for the task.

By Bowen Qin, Yi Xie

rag agents benchmarks

arXiv AI

Jul 29

Addressable Recall Compaction for Long Context-Window Control in AI Agents

arXiv:2607. 25066v1 Announce Type: new Abstract: Long-horizon LLM agents accumulate reasoning traces, actions, and tool observations that can eventually exceed a model's fixed context window.

By Thang Dang, Yuma Ichikawa, Sakina Fatima, Koichi Shirahata

llms agents

arXiv AI

Jul 21

Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration

arXiv:2607. 16848v1 Announce Type: cross Abstract: Long-term memory is becoming a core component of LLM agents, but most memory benchmarks evaluate conversations or compact summaries, while research agents need to restore evidence from full scientific papers.

By Maksim Sheverev, David Finkelstein, Sergey Nikolenko

llms rag agents benchmarks

arXiv Machine Learning

Jun 30

MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation

arXiv:2606. 29914v1 Announce Type: cross Abstract: Agent memory systems are increasingly evaluated against RAG and full-context baselines, but reported gains often mix changes in the memory method with changes in the language model, embedding model, or retrieval pipeline, making it unclear what is actually being measured.

By Kuan Wang

llms rag agents

arXiv AI

Aug 7

OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

arXiv:2607. 28609v2 Announce Type: replace Abstract: Computer-using agents (CUAs) are advancing rapidly across the digital world.

By Qiushi Sun, Kanzhi Cheng, Yian Wang, Bowen Yang, Hang Yan, Liheng Chen, Fangzhi Xu, Zichen Ding, Nuo Chen, Jialin Cao, Xingdong Gong, Zehao Li, Kaiming Jin, Xinfeng Yuan, Zhoumianze Liu, Jingyang Gong, Zhangyue Yin, Jiahui Gao, Zhiyong Wu, Tianbao Xie, Jianbing Zhang, Ben Kao, Lingpeng Kong

llms agents reinforcement-learning multimodal benchmarks safety

arXiv AI

5d ago

FluctlightDB: A Memory Model of Data for AI Agents

arXiv:2608. 12365v1 Announce Type: cross Abstract: For fifty years, data systems have answered two questions.

By Ganesh S

llms rag agents benchmarks

arXiv AI

Jun 3

CoEval: Ranking Language Models for Custom Tasks Without Labeled Data or Trustworthy Benchmarks

arXiv:2606. 03650v1 Announce Type: cross Abstract: Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness.

By Alexander Apartsin, Yehudit Aperstein

llms benchmarks safety

arXiv AI

Jul 3

AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents

arXiv:2607. 02255v1 Announce Type: new Abstract: Memory for a long-horizon LLM agent is a contract about what each future decision is allowed to see.

By Xiangchen Cheng, Yunwei Jiang, Jianwen Sun, Zizhen Li, Chuanhao Li, Xiangcheng Cao, Yihao Liu, Fanrui Zhang, Li Jin, Kaipeng Zhang

llms agents benchmarks

arXiv Machine Learning

Jul 28

When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost

arXiv:2607. 24010v1 Announce Type: new Abstract: Active RAG systems decide when to retrieve external knowledge during generation, making them a budget-sensitive case of agentic RAG and self-adaptive retrieval.

By Pin Qian, Su Wang, Chong Peng, Junxian You, Lifei Liu, Haoran Yu, Yihang Chen, Xiaochong Jiang

rag agents

arXiv AI

Jul 1

ECHO: Prune to act, trace to learn with selective turn memory in agentic RL

arXiv:2606. 31650v1 Announce Type: cross Abstract: Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows.

By Zijun Xie, Binbin Zheng, Enlei Gong, Jihua Liu, Yuyang You, Lingfeng Liu, Jiayao Tang, Guanqun Zhao, Aoqi Hu, Zeyu Chen

agents reinforcement-learning benchmarks

arXiv AI

Jun 10

Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History

arXiv:2606. 09900v1 Announce Type: cross Abstract: Long-term memory is the missing layer for LLM agents: across sessions they forget, and the common workaround -- replaying the whole history into the prompt -- is expensive, slow, and, as distractors accumulate, less accurate.

By Liuyin Wang

llms agents benchmarks

arXiv AI

Aug 11

SodaMem: Evidence-Grounded Temporal Graph Memory for LLM Agents

arXiv:2608. 08055v1 Announce Type: new Abstract: Large language model (LLM) agents that assist users over weeks of conversation must remember what is currently true, not merely what was once said.

By Fengrong Wan, Chengcan Wu, Ningtao Lyu

llms rag agents

Related stories