← Back to all news
arXiv Machine Learning

Exact Record Omission in Delta Attention: A Transport Criterion, Its Cost, and a Replay Certificate

Related stories

arXiv Machine Learning
Sep 10

Can an AI Assistant Really Forget? Auditable Deletion from Addressable Memory

This paper introduces a deletion interface for a pretrained language model, measuring how effectively deleted records are removed from the model’s memory. By retrofitting a support‑vector memory gate into the global attention layers of a frozen Gemma 3, the authors show that deletions can be performed without altering weights and that the resulting state is close to a reference state that never stored the record. Experiments on 4B‑parameter models demonstrate low perplexity impact and strong evidence that deleted content is hard to recover, while larger or smaller models fail to achieve the same guarantees.

By Vishwajith Ramesh
llms
More like this →
arXiv Machine Learning
Oct 23

Delta N-gram: Forward-Only Continual Writing into a Language Model's Native N-gram Memory

Delta N-gram introduces a forward‑only write path into a language model’s native hashed N‑gram table, preserving the pretrained table while adding evidence through a sparse patch Δ. The method uses a 4.1 M‑parameter writer that emits gated targets per event and applies a block‑averaged delta rule to keep each row a convex combination of its old value and the new targets. Experiments on a 0.6 B prototype and Qwen3.8‑Flash‑Next show that the delta writer maintains more document‑specific memory and reduces drift compared to online gradient writes, achieving stable performance over thousands of writes. whyItMatters":"The approach allows continual updates to a language model’s memory without altering its pretrained parameters, enabling efficient, stable, and document‑specific knowledge retention during inference."

By Hao Wu
llms
More like this →
arXiv Machine Learning
Oct 7

What Must Replay Preserve? Separating Correctable Bias from Class Correspondence

The paper investigates what information must be preserved in replay buffers for class‑incremental learning. By treating cached predictions as temporally heterogeneous supervision, the authors separate classes known at storage time from those learned later, and evaluate the impact of deleting logit matching. Experiments on CIFAR‑100 with DER++ show that a simple task‑level offset can largely correct the cost of removing later‑class matching, while the cost of disrupting class correspondence remains.

By BoRen Deng, Xiangyue Ma, Chenglong Li, Xiaoting Du
efficiencysafety
More like this →
arXiv AI
Oct 27

The Price of Scale in Proactive Memory Recall

The paper investigates how a long‑lived assistant decides whether to interrupt a user with a memory recall, framing the decision as a multiple‑testing problem whose difficulty scales with the size of the memory store, N. Experiments on LongMemEval and LoCoMo show that a similarity threshold tuned to a 5% false‑alarm rate at N=8 achieves 87–100% hit rates at N=2048, and theoretical analysis predicts a per‑item false‑positive rate of about –ln(1-α)/N, leading to a power‑law decay of hit rate with N. The study also demonstrates that lexical and semantic channels require thresholds growing like ln N and √ln N, that per‑cue normalisation roughly doubles hit rates, and introduces a cue‑swap conformal gate that achieves near‑label‑tuned performance without labels. whyItMatters":"The work provides a quantitative understanding of how memory‑recall systems must adjust thresholds as their knowledge base grows, informing the design of more efficient and user‑friendly AI assistants."

By Yichuan Chen, Xin Wang
llmsrag
More like this →
arXiv Computation and Language
Sep 23

MoM: Memory of Memory

arXiv:2609.25054v1 Announce Type: new Abstract: For a long-horizon LLM agent, the memory question is not what was once recorded but what \emph{currently holds}. Most designs answer it only indirectly...

By Bowen Qin, Yao Lu
llmsagentsefficiency
More like this →