arXiv AI

User as Engram: Internalizing Per-User Memory as Local Parametric Edits

arXiv:2606. 19172v1 Announce Type: new Abstract: Personal memory in a language model is two problems: content and reasoning skill.

arXiv AI
Jun 24

Repeated Shared Access Enables Grokking, but Edit Propagation Depends on an Addressable Memory

arXiv:2606. 20737v2 Announce Type: replace Abstract: We study factual edit propagation in a controlled synthetic knowledge-graph QA setting using a 2x2 grid that crosses loop recurrence with shared-memory access: a dense transformer (Dense), a looped transformer (Loop), a dense backbone with shared memory (Dense+Mem), and a looped backbone with shared memory (loop-memory coupling, LMC).

By Yanan Niu
arXiv AI
Jul 14

Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models

arXiv:2601. 07372v2 Announce Type: replace-cross Abstract: While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation.

By Xin Cheng, Rui Tian, Wangding Zeng, Damai Dai, Qinyu Chen, Bingxuan Wang, Zhenda Xie, Kezhao Huang, Xingkai Yu, Chengqi Deng, Shangyan Zhou, Chenggang Zhao, Zhewen Hao, Yukun Li, Han Zhang, Zhengyan Zhang, Yixu Wei, M. Y Xu, Huishuai Zhang, Dongyan Zhao, Wenfeng Liang