The paper introduces MCircKE, a mechanistic circuit-based knowledge editing framework for large language models. MCircKE identifies the causal circuits involved in a specific reasoning task and surgically updates parameters only within those circuits, thereby addressing the reasoning gap where edited facts are not used in multi-step reasoning. Experiments on the MQuAKE-series benchmarks show that this approach improves multi-hop reasoning performance after knowledge editing.
By Tianyi Zhao, Yinhan He, Wendy Zheng, Chen Chen
EngramEdit introduces a method for decoupled knowledge updates in large language models using conditional memory architectures. By computing target memory representations that align with updated facts across multiple expressions, it jointly adjusts shared n‑gram embeddings while penalizing frequent ones to preserve unrelated knowledge. Experiments demonstrate near‑perfect editing success, improved multi‑hop reasoning, and strong performance under chain‑of‑thought prompting, all while maintaining general capabilities.
By Hongru Cai, Ran Wei, Wenjie Wang, Chengfa Wu, Ning Song, Yongqi Li, Wenjie Li
Ladders-of-Thought (LoT) is a framework that enhances reasoning in small- to mid-scale large language models by automatically generating easier variants of reasoning problems and organizing them into difficulty buckets. It uses a self‑evolving bandit scheduler to adaptively allocate training, improving performance across math and multi‑hop reasoning tasks on 1–8 B models. LoT achieves significant gains (e.g., +32 pp on AddSub, +16 pp on QASC) and converges faster than staged curricula.
By Minghui Liu, Thomas Magelinski, Dehao Yuan, Qi Yu, Furong Huang
The paper introduces Golden-GRPO Injection (GRIN), a three-stage self‑learning framework that uses a mixed‑policy reinforcement learning algorithm to inject knowledge into large language models. GRIN injects a golden answer to provide learning signals even when on‑policy rollouts fail on novel facts, and is evaluated on two new document‑level benchmarks—Blank and Counter—that test novel acquisition and counterfactual overwrite. Experiments show that mixed‑policy RL enables knowledge absorption beyond what supervised fine‑tuning can achieve, with GRIN outperforming SFT and other RL baselines on harder question types while matching them on basic fact recall.
By Zhibo Hou, Fan Zhao, Zhiyu An, Wan Du
arXiv:2608. 11660v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world.
By Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, Jing Gao
arXiv:2511.05933v3 Announce Type: replace-cross
Abstract: Reinforcement learning (RL) is often credited with improving reasoning at the expense of factual knowledge. We instead find that reasoning mo...
By Renfei Zhang, Manasa Kaniselvan, Rylan Schaeffer, Niloofar Mireshghallah
The paper introduces FOVEATED, a plug‑and‑play framework that improves atomic‑fact recall in unstructured knowledge editing (UKE) for large language models. By randomly shifting Rotary Position Embedding (RoPE) positions during editing, FOVEATED creates focused views of each sentence, counteracting the context‑reliance problem where edited LLMs reproduce passages but fail to recall individual facts. Experiments show consistent gains across five editors, two LLM backbones, and three benchmarks.
By Ding Wu, Ye Zhang, Haoyu Wang, Tianci Liu
arXiv:2607. 14049v1 Announce Type: new Abstract: The emergence of Chain-of-Thought (CoT) reasoning has significantly enhanced the ability of large language models (LLMs) to tackle complex, multi-step tasks.
By Hefeng Zhou, Jinxuan Zhang, Jiong Lou, Yuxin Liu, Chaochao Lu, Jingjing Qu, Jie Li
arXiv:2607. 00341v1 Announce Type: cross Abstract: Large language models achieve strong performance on many reasoning tasks when allowed to externalize intermediate steps as Chain-of-Thought (CoT).
By Hengyu Fu, Tianyu Guo, Zixuan Wang, Hanlin Zhu, Jason D. Lee, Jiantao Jiao, Stuart Russell, Song Mei
The paper introduces Multi-Objective In-context Knowledge Editing (MO‑IKE), a reinforcement learning framework that treats prompt construction for knowledge editing as a constrained Markov decision process. MO‑IKE jointly optimizes three competing objectives—reliability, generality, and specificity—by training a dynamic retriever to balance these goals and produce globally coherent prompts. Experiments on Llama‑3.2 show that MO‑IKE raises edit success from 85.0 % to 92.0 %, improves paraphrase consistency from 77 % to 79 %, and boosts retention rate by 23 % compared to earlier RL‑based methods.
By Xuzhong Wang, Maiqi Jiang, Tejal Nair, Girija Bhusal, Yanfu Zhang, Haipeng Chen
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
By Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, Jun Wang
The paper introduces RD-Forget, a training‑free framework that separates what a persistent language agent stores from what it uses at answer time. It keeps a source archive of all observations while a query‑conditioned memory view filters evidence relevant to the current question, using a frozen language‑model curator to group facts into semantic slots and preserve multi‑hop relations. The approach employs rate‑distortion principles to stay within a memory budget and demonstrates improvements across conversational memory, knowledge updating, fact consolidation, long‑context reasoning, and personalization tasks.
By Yuhang Li, Yuchen Li