The paper introduces FOVEATED, a plug‑and‑play framework that improves atomic‑fact recall in unstructured knowledge editing (UKE) for large language models. By randomly shifting Rotary Position Embedding (RoPE) positions during editing, FOVEATED creates focused views of each sentence, counteracting the context‑reliance problem where edited LLMs reproduce passages but fail to recall individual facts. Experiments show consistent gains across five editors, two LLM backbones, and three benchmarks.
By Ding Wu, Ye Zhang, Haoyu Wang, Tianci Liu
arXiv:2608. 11660v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world.
By Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, Jing Gao
The paper introduces Style‑Debiased DPO (SD‑DPO), a method that refines large language models’ ability to retrieve stored knowledge by using preference optimization that corrects for style differences while preserving factual accuracy. SD‑DPO evaluates on the EntiGraph storing‑side framework and outperforms baseline CPT on the QuALITY reading‑comprehension benchmark, achieving higher accuracy with far fewer training tokens. In a knowledge‑editing setting (AToKE), SD‑DPO attains an overall accuracy of 0.982, correctly answering queries with either new or old facts based on the requested time period.
By Takayuki Yamamoto, Daisuke Kawahara
arXiv:2511. 20892v4 Announce Type: replace Abstract: Large language models (LLMs) often produce incorrect or outdated content after being employed.
By Xuyuan Liu, Shengyu Chen, Xinshuai Dong, Yanchi Liu, Xujiang Zhao, Haoyu Wang, Yujun Yan, Haifeng Chen, Zhengzhang Chen
The paper introduces MCircKE, a mechanistic circuit-based knowledge editing framework for large language models. MCircKE identifies the causal circuits involved in a specific reasoning task and surgically updates parameters only within those circuits, thereby addressing the reasoning gap where edited facts are not used in multi-step reasoning. Experiments on the MQuAKE-series benchmarks show that this approach improves multi-hop reasoning performance after knowledge editing.
By Tianyi Zhao, Yinhan He, Wendy Zheng, Chen Chen
The paper introduces VAKE, a two‑stage reinforcement‑learning framework that activates latent factual knowledge in large language models. In the Priming stage, the model explicitly inserts bridging triples into an insufficient subgraph, guided by rewards from a frozen model’s answers. The Reasoning stage then trains the model to answer from the original input, demonstrating that the elicitation capability transfers to implicit reasoning and consistently outperforms baselines across multiple benchmarks and model sizes.
By Zuocheng Ying, Yang Yang, Yumou Wu, Chuanbo Zhu, Jiarui Wang, Ziqi Wu, Jingming Cai, Junqing Yu, Zikai Song