arXiv:2607. 11020v1 Announce Type: cross Abstract: Continual learning promises a language model that keeps acquiring knowledge after training, with each new fact written into its weights.
By Charles O'Neill
arXiv:2610.27580v1 Announce Type: cross
Abstract: Factual recall is a lookup: some representation inside the network serves as the key, and what that key is made of decides which queries can retrieve...
By Yuchen Guo, Zhuoyuan Mao, Yiwen Guo
arXiv:2608. 11660v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world.
By Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, Jing Gao
The paper introduces a retrieval‑augmented language model that resolves disagreements between retrieved context and the model’s own knowledge by incorporating a prior over the reliability of the context rather than a fixed rule. Experiments on the GRACE benchmark show that this prior‑based approach outperforms existing baselines by up to 26 points, achieving higher overall scores on both Qwen3.5‑35B‑A3B and Llama‑3.1‑8B. The method also demonstrates superior performance on the ClashEval benchmark, confirming the theoretical advantage of evidence‑based conflict resolution.
By Minh Pham Van, Cong Tran
The study investigates how well language models can generalize from fine‑tuned facts by distinguishing active facts—those that support further learning—from inert facts. Using a novel relabeling experiment (e.g., mapping Paris → Europe to Paris → crab), the authors fine‑tune models on a subset of subjects and test whether the learned mapping extends to unseen subjects sharing the same original answer. Across eleven models and 70 relations, they find a strong correlation (Pearson r = 0.87–0.92) between the linear readability of the original relation in pretrained representations and the model’s ability to generalize, a relationship that persists after controlling for various factors and remains robust to label randomization and semantic category changes.
By Faridoun Mehri, Hinrich Sch\"utze
arXiv:2602. 02470v2 Announce Type: replace Abstract: Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal curse" -- when trained on forward knowledge data of the form "$A \rightarrow B$" (e.
By Xutao Ma, Yixiao Huang, Hanlin Zhu, Somayeh Sojoudi