RubricEM is a unified reinforcement learning framework designed for deep research agents that plan, search, evaluate evidence, and synthesize long‑form reports. It introduces rubric‑guided policy decomposition, combining supervised fine‑tuning to scaffold agents with rubrics, stage‑structured GRPO for fine‑grained credit assignment, and a reflection meta‑policy for experience reuse. The resulting RubricEM‑8B model outperforms larger open models on four long‑form research benchmarks and approaches the performance of proprietary deep‑research systems.
By Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister
All-in-One Memory (AiO-Mem) is a reinforcement learning framework that unifies memory retrieval, writing, and stopping decisions into a single reward space. It compiles diverse histories into traceable multi‑view events, uses a unified policy actor to generate interaction trajectories, and employs a trajectory‑conditioned process reward model trained with supervised fine‑tuning and DPO. Experiments on long‑context memory benchmarks show AiO‑Mem consistently improves answer accuracy compared to static memory systems and other RL‑trained agents.
By Bingbing Wang, Jingjie Lin, Weiming Qiao, Jing Li, Ruifeng Xu
arXiv:2607. 13591v1 Announce Type: cross Abstract: Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.
By Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu, Levina Li, Dong Liu, Xiao Liang, Rui Sun, Yubei Li, Edward Sun, Haozheng Luo, Zhaolu Kang, Aylin Caliskan, Kai-Wei Chang, Ying Nian Wu
Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and lengthy documents. However, existing memory methods...
arXiv:2607. 21106v1 Announce Type: new Abstract: Effective memory is crucial for LLM agents, yet constructing it effectively remains challenging.
By Qinfeng Li, Yuntai Bao, Xinyan Yu, Hongze Chen, Wenqi Zhang, Xuhong Zhang
Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks. Yet nearly all existing approaches, from graph-structured memories to reflective insight stores, access memory through fixed, hand-designed heuristics.
arXiv:2607. 21106v2 Announce Type: replace Abstract: Effective memory is crucial for LLM agents, yet constructing it effectively remains challenging.
By Qinfeng Li, Yuntai Bao, Xinyan Yu, Hongze Chen, Yanmin Liu, Wenqi Zhang, Xuhong Zhang
arXiv:2606. 10616v1 Announce Type: new Abstract: Long-horizon language agents accumulate observations, reasoning traces, and retrieved facts that exceed their finite context windows, making memory retention a fundamental resource-allocation problem.
By Qingcan Kang, Liu Mingyang, Shixiong Kai, Kaichao Liang, Tao Zhong, Mingxuan Yuan
arXiv:2608. 10108v1 Announce Type: new Abstract: Long-horizon agents accumulate trajectories spanning hundreds of interleaved reasoning, action, and observation steps, where answering a query may depend on evidence buried far back in the history.
By Beidi Zhao, Yaoqi Chen, Yuru Feng, Menghao Li, Qianxi Zhang, Baotong Lu, Jianan Lu, Zhirui Wang, Xinjiang Wang, Shusen Xu, Zengzhong Li, Xiaoxiao Li, Qi Chen
arXiv:2606. 06787v1 Announce Type: new Abstract: Large Language Models (LLMs) show promise as tool-using agents but remain limited in long-horizon tasks that require remembering, organizing, and reusing knowledge.
By Runzhe Wang, Huilin Lu, Shengjie Liu, Li Dong, Jason Zhu
arXiv:2510. 05592v2 Announce Type: replace Abstract: Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios.
By Zhuofeng Li, Haoxiang Zhang, Seungju Han, Sheng Liu, Jianwen Xie, Yu Zhang, Yejin Choi, James Zou, Pan Lu
arXiv:2606. 28434v1 Announce Type: cross Abstract: Long-horizon software engineering agents often need to manage lengthy and noisy interaction histories under limited context budgets.
By Shuzheng Gao, Wenhao Zeng, Zhaojian Yu, Jianqiao Wangni, Chaozheng Wang, Kai Cai, Shilin He, Michael R. Lyu