arXiv Machine Learning

From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection

The paper introduces Golden-GRPO Injection (GRIN), a three-stage self‑learning framework that uses a mixed‑policy reinforcement learning algorithm to inject knowledge into large language models. GRIN injects a golden answer to provide learning signals even when on‑policy rollouts fail on novel facts, and is evaluated on two new document‑level benchmarks—Blank and Counter—that test novel acquisition and counterfactual overwrite. Experiments show that mixed‑policy RL enables knowledge absorption beyond what supervised fine‑tuning can achieve, with GRIN outperforming SFT and other RL baselines on harder question types while matching them on basic fact recall.

arXiv AI

Aug 25

DeepRefine: Agentic Knowledge Refinement via Reinforcement Learning

DeepRefine is a reinforcement learning framework that improves the quality of pre‑constructed structured knowledge bases—such as knowledge graphs or LLM‑Wikis—by engaging in multi‑turn interactions with the base. It performs abductive diagnosis to locate defects, then applies targeted refinement actions to incrementally update the knowledge base. The system uses a Gain‑Beyond‑Draft reward to train its refinement policy end‑to‑end, achieving consistent downstream performance gains over strong baselines.

By Haoyu Huang, Jiaxin Bai, Shujie Liu, Yang Wei, Huihao Jing, Hong Ting Tsang, Yisen Gao, Zhongwei Xie, Yufei Li, Yangqiu Song

llms rag agents reinforcement-learning

arXiv AI

Jun 16

Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift

arXiv:2606. 15734v1 Announce Type: cross Abstract: Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters can accumulate weight drift, potentially causing catastrophic forgetting and degrading general capabilities.

By Weihang Su, Jiacheng Kang, Jingyan Xu, Qingyao Ai, Jianming Long, Hanwen Zhang, Bangde Du, Xinyuan Cao, Min Zhang, Yiqun Liu

rag

arXiv Machine Learning

Sep 30

Learning What to Remember: Long-horizon Counterfactual Memory Optimization

arXiv:2609.37930v1 Announce Type: cross Abstract: Persistent textual memory allows language models to carry information across long interactions, but learning what to remember is fundamentally a cred...

By Jiaming Tang, Mingyan Liu, Armin Sarabi

llms

arXiv AI

Jun 19

Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

arXiv:2510. 21978v2 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning and has become a standard post-training paradigm for contemporary language and vision-language models.

By Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu, Deren Lei

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jul 14

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts

arXiv:2508. 10123v3 Announce Type: replace-cross Abstract: Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT).

By Maxime Heuillet, Yufei Cui, Boxing Chen, Audrey Durand, Prasanna Parthasarathi

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 28

MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents

arXiv:2607. 24097v1 Announce Type: new Abstract: Memory-augmented LLM agents typically answer queries by retrieving relevant memories and feeding them directly to an answer model.

By Yiwen Ma, Songjun Tu, Qichao Zhang, Dong Li, Linjing Li, Dongbin Zhao

llms agents reinforcement-learning benchmarks

arXiv Computation and Language

Sep 1

REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation

arXiv:2608.30627v1 Announce Type: new Abstract: As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token pred...

By Haoran Que, Jiajun Shi, Ting Huang, Renming Pang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Shen Yan, Wei Ye, Shikun Zhang

benchmarks

arXiv AI

Jul 14

CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps

arXiv:2607. 10562v1 Announce Type: new Abstract: Evaluating the multi-hop reasoning capabilities of large language models remains a significant challenge.

By JungMin Yun, JuneHyoung Kwon, YoungBin Kim

llms nlp

arXiv AI

Sep 30

CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory

CoEM introduces a Commit-on-Evidence Memory system that learns when to compress source evidence into compact memory facts while preserving potentially useful excerpts verbatim in a pending set. The system uses a learned policy to decide whether to promote, retain, or discard each pending excerpt as new context arrives, and a frozen verifier ensures only supported facts are committed. Reinforcement learning trains this policy with step-level evidence rewards and final answer rewards, leading to consistent improvements in long-context reasoning, achieving 10.4–11.4 F1 points over the strongest baseline on 6,400-document inputs.

By Jingguang Li, Yebo Wu, Zuyi Guo, Kailang Ma, Xianjie Dai, Han Zheng, Benwang Chen, Li Li, Can Rong, Heye Huang

llms reinforcement-learning

arXiv AI

Jun 16

Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories

arXiv:2602. 02028v2 Announce Type: replace Abstract: Enabling artificial intelligence systems, particularly large language models, to update knowledge and flexibly apply it during reasoning remains a central challenge.

By Ya Gao, Kalle Kujanp\"a\"a, Pekka Marttinen, Harri Valpola, Alexander Ilin

llms efficiency

arXiv AI

Aug 24

SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation

arXiv:2509.20377v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has significantly improved the performance of large language models (LLMs) on knowledge-intensive tasks...

By Tomoaki Isoda

llms rag nlp reinforcement-learning benchmarks safety

arXiv AI

Sep 11

What Should an Agent Forget? Separating What Is Stored from What Is Used

The paper introduces RD-Forget, a training‑free framework that separates what a persistent language agent stores from what it uses at answer time. It keeps a source archive of all observations while a query‑conditioned memory view filters evidence relevant to the current question, using a frozen language‑model curator to group facts into semantic slots and preserve multi‑hop relations. The approach employs rate‑distortion principles to stay within a memory budget and demonstrates improvements across conversational memory, knowledge updating, fact consolidation, long‑context reasoning, and personalization tasks.

By Yuhang Li, Yuchen Li

agents

Related stories