The paper introduces Legible, a fine‑tuning technique that aligns a language model’s internal representation of a newly added fact with the model’s own reading of that fact, thereby restoring the model’s ability to perform two‑hop reasoning. By adding a loss term that forces the bridge phrase to match the model’s internal entity representation, Legible improves the model’s composition of learned facts and significantly boosts bridge accuracy across several large‑language‑model families. The method achieves these gains without requiring multi‑hop training data, outperforming prior approaches such as CaKE and RippleEdits on benchmark tests.
arXiv:2607. 11020v1 Announce Type: cross Abstract: Continual learning promises a language model that keeps acquiring knowledge after training, with each new fact written into its weights.
arXiv:2610.27580v1 Announce Type: cross Abstract: Factual recall is a lookup: some representation inside the network serves as the key, and what that key is made of decides which queries can retrieve...
arXiv:2608. 11660v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world.
The paper introduces a retrieval‑augmented language model that resolves disagreements between retrieved context and the model’s own knowledge by incorporating a prior over the reliability of the context rather than a fixed rule. Experiments on the GRACE benchmark show that this prior‑based approach outperforms existing baselines by up to 26 points, achieving higher overall scores on both Qwen3.5‑35B‑A3B and Llama‑3.1‑8B. The method also demonstrates superior performance on the ClashEval benchmark, confirming the theoretical advantage of evidence‑based conflict resolution.
The study investigates how well language models can generalize from fine‑tuned facts by distinguishing active facts—those that support further learning—from inert facts. Using a novel relabeling experiment (e.g., mapping Paris → Europe to Paris → crab), the authors fine‑tune models on a subset of subjects and test whether the learned mapping extends to unseen subjects sharing the same original answer. Across eleven models and 70 relations, they find a strong correlation (Pearson r = 0.87–0.92) between the linear readability of the original relation in pretrained representations and the model’s ability to generalize, a relationship that persists after controlling for various factors and remains robust to label randomization and semantic category changes.
arXiv:2602. 02470v2 Announce Type: replace Abstract: Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal curse" -- when trained on forward knowledge data of the form "$A \rightarrow B$" (e.
arXiv:2608.20873v1 Announce Type: new Abstract: Every way of teaching a deployed language model something new -- full fine-tuning, adapter merging, model editing -- replaces the released checkpoint,...
arXiv:2606. 19172v1 Announce Type: new Abstract: Personal memory in a language model is two problems: content and reasoning skill.
The paper distinguishes between active facts, which help a language model generalize beyond mere recall, and inert facts, which do not. By relabeling answers to a known relation (e.g., Paris → Europe becomes Paris → crab) and fine‑tuning on a subset of subjects, the authors test whether the model can transfer the new label to unseen subjects sharing the same original answer. Across eleven models and 70 relations, the linear readability of the original relation in pretrained representations strongly predicts this generalization, with a Pearson correlation of 0.87–0.92, even after controlling for various factors.
The paper introduces “Rel-Bloom,” a multilingual diagnostic benchmark of 450 multiple-choice questions that tests language models’ ability to reason over relational facts supplied directly in prompts. It evaluates 43 models across six languages and two multilingual settings, revealing that accuracy drops sharply from Apply to Analyze levels, that Evaluate items remain a bottleneck even for high-performing models, and that difficulty rankings are consistent across languages. The study also finds that counterfactual contexts are generally harder than factual ones, and that removing entity identities can sometimes improve higher-level performance.
The study investigates whether the reasoning steps a language model writes are causally responsible for its answers. Using a causal intervention method on the activation stream, the authors find that for Qwen3-4B, about 77% of stated steps are causally load‑bearing, while behavioral tests overestimate this by roughly 11 percentage points. The faithfulness of reasoning decreases with model size and depth of reasoning, especially for the smaller Qwen3-1.7B.
TraceForge introduces a two‑stage training method that generates reasoning traces in the style of a target model family, using supervised learning followed by adversarial refinement. The system produces traces that are judged as authentic with scores comparable to real traces, and when applied to teacher answers, it improves performance on coding benchmarks such as HumanEval+ and MBPP+. The approach addresses the limitation of existing models that only return answers without detailed reasoning, enabling more transparent and learnable outputs.