arXiv Machine Learning

From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection

The paper introduces Golden-GRPO Injection (GRIN), a three-stage self‑learning framework that uses a mixed‑policy reinforcement learning algorithm to inject knowledge into large language models. GRIN injects a golden answer to provide learning signals even when on‑policy rollouts fail on novel facts, and is evaluated on two new document‑level benchmarks—Blank and Counter—that test novel acquisition and counterfactual overwrite. Experiments show that mixed‑policy RL enables knowledge absorption beyond what supervised fine‑tuning can achieve, with GRIN outperforming SFT and other RL baselines on harder question types while matching them on basic fact recall.

arXiv AI

Aug 25

DeepRefine: Agentic Knowledge Refinement via Reinforcement Learning

DeepRefine is a reinforcement learning framework that improves the quality of pre‑constructed structured knowledge bases—such as knowledge graphs or LLM‑Wikis—by engaging in multi‑turn interactions with the base. It performs abductive diagnosis to locate defects, then applies targeted refinement actions to incrementally update the knowledge base. The system uses a Gain‑Beyond‑Draft reward to train its refinement policy end‑to‑end, achieving consistent downstream performance gains over strong baselines.

By Haoyu Huang, Jiaxin Bai, Shujie Liu, Yang Wei, Huihao Jing, Hong Ting Tsang, Yisen Gao, Zhongwei Xie, Yufei Li, Yangqiu Song

llms rag agents reinforcement-learning

arXiv AI

Jun 16

Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift

arXiv:2606. 15734v1 Announce Type: cross Abstract: Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters can accumulate weight drift, potentially causing catastrophic forgetting and degrading general capabilities.

By Weihang Su, Jiacheng Kang, Jingyan Xu, Qingyao Ai, Jianming Long, Hanwen Zhang, Bangde Du, Xinyuan Cao, Min Zhang, Yiqun Liu

rag

arXiv AI

Jun 19

Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models

arXiv:2510. 21978v2 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning and has become a standard post-training paradigm for contemporary language and vision-language models.

By Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu, Deren Lei

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jul 14

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts

arXiv:2508. 10123v3 Announce Type: replace-cross Abstract: Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT).

By Maxime Heuillet, Yufei Cui, Boxing Chen, Audrey Durand, Prasanna Parthasarathi

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 28

MemChain: Learning Interpretable Memory Traces for Memory-Augmented LLM Agents

arXiv:2607. 24097v1 Announce Type: new Abstract: Memory-augmented LLM agents typically answer queries by retrieving relevant memories and feeding them directly to an answer model.

By Yiwen Ma, Songjun Tu, Qichao Zhang, Dong Li, Linjing Li, Dongbin Zhao

llms agents reinforcement-learning benchmarks

arXiv Computation and Language

3d ago

REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation

arXiv:2608.30627v1 Announce Type: new Abstract: As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token pred...

By Haoran Que, Jiajun Shi, Ting Huang, Renming Pang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Shen Yan, Wei Ye, Shikun Zhang

benchmarks

arXiv AI

Jul 14

CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps

arXiv:2607. 10562v1 Announce Type: new Abstract: Evaluating the multi-hop reasoning capabilities of large language models remains a significant challenge.

By JungMin Yun, JuneHyoung Kwon, YoungBin Kim

llms nlp

arXiv AI

Jun 16

Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories

arXiv:2602. 02028v2 Announce Type: replace Abstract: Enabling artificial intelligence systems, particularly large language models, to update knowledge and flexibly apply it during reasoning remains a central challenge.

By Ya Gao, Kalle Kujanp\"a\"a, Pekka Marttinen, Harri Valpola, Alexander Ilin

llms efficiency

arXiv AI

Aug 24

SKILL-RAG: Self-Knowledge Induced Learning and Filtering for Retrieval-Augmented Generation

arXiv:2509.20377v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has significantly improved the performance of large language models (LLMs) on knowledge-intensive tasks...

By Tomoaki Isoda

llms rag nlp reinforcement-learning benchmarks safety

arXiv AI

Jul 21

It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.

By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang

llms reinforcement-learning benchmarks

arXiv AI

Jul 28

MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models

arXiv:2607. 22556v1 Announce Type: new Abstract: Continual learning (CL) is essential for small language models (SLMs) to adapt to evolving real-world needs in resource-constrained deployments.

By Dong Li, Yanchi Liu, Xujiang Zhao, Wei Cheng, Zhengzhang Chen, Xintao Wu, Zhong Chen, Chen Zhao, Haifeng Chen

llms

arXiv AI

Jul 22

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

arXiv:2607. 19345v1 Announce Type: cross Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier.

By Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

llms reinforcement-learning benchmarks

Related stories