The paper introduces Online Meta-Control of Experience (OMCE), a framework that updates an agent’s experience memory and learns a meta-control policy during test-time adaptation. OMCE uses a lightweight contextual-bandit meta-controller to choose among four cognitive actions—DIRECT, EPISODE, GUIDE, and DISTILL—based on compact retrieval signals, thereby deciding when to invoke experience, which memory format to use, and when to consolidate new guidance. Experiments on four online adaptation benchmarks across four backbones show that OMCE can improve first-attempt success by up to 20.5 percentage points over the strongest baseline while reducing per-task token use by a factor of 5.7 to 62.5 relative to ACE with GPT‑5.2, and controlled ablations confirm the benefits of learned routing and ongoing policy adaptation.
By Zeyuan Zhao, Yuanyuan Shi, Peng Yu, Lingyue Fu, Shao Zhang, Ying Wen
arXiv:2607. 13591v1 Announce Type: cross Abstract: Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.
By Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu, Levina Li, Dong Liu, Xiao Liang, Rui Sun, Yubei Li, Edward Sun, Haozheng Luo, Zhaolu Kang, Aylin Caliskan, Kai-Wei Chang, Ying Nian Wu
arXiv:2607. 29241v1 Announce Type: cross Abstract: Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes.
By Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao, Shuwen Kang, Chi Lu, Wenjin Wu, Peng Jiang
Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks. Yet nearly all existing approaches, from graph-structured memories to reflective insight stores, access memory through fixed, hand-designed heuristics.
Adaptive Memory Index Evolution for Agents (AMIE) is a lightweight plug‑in layer that evolves per‑memory access state from deployment retrievals without altering stored content or retraining the host memory system. It distills a semantic teacher’s keep/suppress judgments into a retrieval‑conditioned policy and accumulates selective actions into bounded state that reweights subsequent host retrievals. Across five retrieval‑based hosts and three benchmarks, trained AMIE improves downstream F1 or accuracy in all 15 comparisons (0.70–3.37 percentage points) and improves retrieval in all 10 comparisons, demonstrating that persistent memory access can be adapted independently of memory content.
By Yuntong Hu, Tongli Su, Yuanfu Sun, Shuo Ma, Qiaoyu Tan, Liang Zhao
arXiv:2608.21946v1 Announce Type: cross
Abstract: Reinforcement learning with outcome-based objectives such as GRPO enables LLM-based agents to solve complex, long-horizon tasks, yet the reusable exp...
By Can Xie, Yuyi Zhou, Wen Yang, Ziyi zhang, Siyao Song, Yingzhuo Deng, Shuo Ren, Jiajun Zhang
arXiv:2606. 04536v1 Announce Type: new Abstract: Existing memory-augmented LLM agents store past experience exclusively in prompt space, as textual summaries or retrieved passages, while keeping model parameters frozen throughout a rollout.
By Tao Ren, Weiyao Luo, Hui Yang, Rongzhi Zhu, Xiang Huang, Yuchuan Wu, Bingxue Chou, Jieping Ye, Jiafeng Liang, Yongbin Li, Yijie Peng
The paper introduces TokenWorth, a lightweight framework that learns the value of past experiences for self-improving agents operating under strict resource constraints. By using sparse execution feedback, TokenWorth guides both retrieval and memory management without updating the core model, enabling agents to adaptively reuse only the most beneficial experiences. Experiments on LifelongAgentBench show that TokenWorth outperforms existing lifelong-memory methods in task success while minimizing output tokens within the given budgets.
By Guannan Lai, Zi-Yang Bai, Jun-Peng Jiang, Han-Jia Ye
arXiv:2606. 02373v1 Announce Type: new Abstract: Search agents are often trained as policies over growing transcripts: the model must decide how to search while also remembering what it has seen, which evidence is useful, which constraints remain open, and which claims have actually been checked.
By Pengcheng Jiang, Zhiyi Shi, Kelly Hong, Xueqiang Xu, Jiashuo Sun, Jimeng Sun, Hammad Bashir, Jiawei Han
The paper introduces Just-in-Time Memory (JitMem), a system that defers memory curation until a task is read, allowing a curator to synthesize task‑specific memory payloads based on the current query. Unlike traditional write‑time curation, JitMem retains raw trajectories and trains the curator using immediate task success, avoiding long‑horizon credit‑assignment issues. Experiments on ALFWorld, WebShop, and τ²‑bench show JitMem consistently outperforms both no‑memory agents and existing write‑time memory methods, with improvements of up to 16.3 absolute success‑rate points.
whyItMatters":"By curating memory at read time, JitMem enables more effective, task‑adaptive recall that directly improves agent performance across diverse benchmarks."
By Yefan Zhou, Yang Li, Zeyu Leo Liu, Semih Yavuz, Shafiq Joty
The paper introduces OPSD, a method that uses on‑policy self‑distillation to train a context‑free student model from a teacher conditioned on user memory, combined with a verify‑then‑regenerate router that directs queries to either weight or text memory. OPSD enables weight memory to compete with or surpass traditional text‑based memory systems on the HaluMem benchmark, reducing token usage per query. The study also explores how different training recipes influence weight memory performance and offers guidance for integrating it with retrieval in persistent, multi‑session agent deployments.
By Zeyu Fang, Jun Yin, Yongsheng Mei, Xu Han, Chaitali Joshi, Zhongkai Sun, Jie Hao, Bin Guo, Chengyuan Ma, Benjamin Z. Yao, Chenlei Guo
arXiv:2608. 12720v1 Announce Type: cross Abstract: While Large Language Model (LLM) agents increasingly rely on long-term memory for persistent interactions, the retrieval mechanisms governing this memory are rarely treated as evolvable components.
By Haolong Chen, Liang Zhang, Zhuo Li, Lei Xue, Guanrxu Zhu