arXiv Computation and Language By Jonathan Zheng, Zirui Shao, Alan Ritter, Wei Xu

Parallel Universes for Temporal Evaluation and Knowledge Updating in LLMs

Read the original on arXiv Computation and Language →

The paper introduces PARALLELEVENTS, a benchmark that simulates realistic future event trajectories to evaluate knowledge updates in large language models without contaminating data or conflicting with existing knowledge. It also presents SYNAPSE, a training framework that uses model-generated synthetic data for instruction tuning, enabling scalable integration of new knowledge. SYNAPSE achieves a 14.23‑point improvement in QA accuracy over previous methods, demonstrating the effectiveness of simulation‑based synthetic training for coherent knowledge insertion.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Oct 27

DynEdit: Dynamic Entropy-Guided Sequential Editing for Large Language Models

DynEdit is a dynamic framework for editing long-sequence knowledge in large language models. It uses token-level entropy to guide non-uniform sliding windows, focusing on high-entropy, information-dense regions that are harder to edit. Experiments on three benchmarks and two backbone models show DynEdit outperforms baselines, achieving up to +14.0 BLEU points on challenging paraphrase queries and near-saturated semantic scores across diverse knowledge domains.

By Jinhu Fu, Yan Bai, Yihang Lou, Li Sun, Jiahong Wu, Xiangxiang Chu, Sen Su
Hugging Face Trending Papers
Jul 29

ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models

Large language models (LLMs) have demonstrated strong capabilities in knowledge acquisition and reasoning, yet their ability to retain previously acquired knowledge under repeated updates remains insufficiently understood. Existing evaluation paradigms primarily focus on single-step reasoning or static knowledge editing, which fail to capture the temporal dynamics of knowledge retention and degradation during continual model modification.