arXiv AI
Factual Access States in Diffusion Language Models: From Mechanism to Knowledge Editing and Retrieval
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 13
Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing
arXiv:2608. 11660v1 Announce Type: cross Abstract: Large language models (LLMs) achieve remarkable performance across natural language tasks, yet they are trained on static corpora and their knowledge quickly becomes outdated in a fast-changing world.
arXiv AI
Oct 1
Frozen Memory Is Not Enough: Rethinking External Memory as Extraction
arXiv:2608.17050v3 Announce Type: replace-cross Abstract: Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access...
arXiv AI
Oct 5
Improving Atomic-Fact Recall via Focused Views in Unstructured Knowledge Editing
The paper introduces FOVEATED, a plug‑and‑play framework that improves atomic‑fact recall in unstructured knowledge editing (UKE) for large language models. By randomly shifting Rotary Position Embedding (RoPE) positions during editing, FOVEATED creates focused views of each sentence, counteracting the context‑reliance problem where edited LLMs reproduce passages but fail to recall individual facts. Experiments show consistent gains across five editors, two LLM backbones, and three benchmarks.
arXiv AI
Jul 24
Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing
arXiv:2607. 20433v1 Announce Type: cross Abstract: While language models remain frozen at their training state, the world evolves continuously.
arXiv AI
Jun 18
User as Engram: Internalizing Per-User Memory as Local Parametric Edits
arXiv:2606. 19172v1 Announce Type: new Abstract: Personal memory in a language model is two problems: content and reasoning skill.
arXiv AI
Oct 2
The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
The paper investigates how large language models (LLMs) encode and retrieve facts based on their order of mention in a context. By measuring changes in hidden states when shifting questions from one fact to another, the authors identify an "ordinal vector" that points to a fact’s position in the list rather than its name. Across several models (Qwen, Gemma, Llama), these ordinal addresses are ordered, steerable, low‑rank, and emerge in mid‑layers, suggesting that LLMs locate facts by their mention order, mirroring aspects of human recall.