arXiv AI

When to Remember, When to Abstain: Category-Conditioned Retention for Reliable Agent Memory

The paper proposes a category‑conditioned retention strategy for agent memory, arguing that a single global confidence threshold is insufficient to balance reliability across different assertion types. By applying stricter retention bars to value assertions while allowing more liberal retention for other categories, the authors demonstrate a 36% relative reduction in unsupported value claims and a 13‑percentage‑point increase in coverage compared to a global threshold. The study is evaluated on a cold‑start memory pipeline with 100 synthetic personas, showing that selective, category‑aware retention improves overall memory reliability.

arXiv AI

1d ago

The Right Memory in the Wrong Context: Verifying Retrieval Admissibility in Long-Term Agent Memory

The paper presents a retrieval‑admissibility verification framework for long‑term memory agents, classifying each memory‑query pair as admissible, inadmissible, or unresolved. It evaluates the framework on public benchmarks (RHELM and MemOps), showing improved anchor recall and reduced exact similarity errors, while also revealing that existing verifiers miss certain inadmissible exposures. The study highlights the need for separate checks on candidate support, admissibility, prompt exposure, and answer disclosure to ensure safe memory retrieval.

By Zi Wang, Xingqiao Wang, Emmanuel Addai, Devika Ambekar, Xiaowei Xu

agents benchmarks

arXiv AI

Jun 30

Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident Facts

arXiv:2606. 29279v1 Announce Type: cross Abstract: LLM agents carry conclusions across steps and sessions in compressed memory, and memory products (e.

By Alex Kwon

llms agents

arXiv AI

Jul 17

When Does Belief-Based Agent Memory Help? Reliability-Conditional Updating and Provenance-Capped Poisoning Defense

arXiv:2606. 22030v2 Announce Type: replace Abstract: We investigate when belief-based memory actually improves large language model (LLM) agents.

By Pranav Singh

llms agents robotics benchmarks

arXiv Computation and Language

Aug 21

Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents

arXiv:2608. 19564v1 Announce Type: new Abstract: Persistent memory can personalize an LLM agent, but an incorrect durable update can silently distort future behavior.

By Baichuan Li, Junyi Yao, Zihao Zheng

llms agents

arXiv AI

Jun 16

Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations

arXiv:2606. 15903v1 Announce Type: cross Abstract: Where an LLM sits in an agent memory pipeline -- between the recall plane that retrieves stored facts (extensively benchmarked) and the control plane that mutates them via supersede, release, purge (largely untested) -- shapes which forgetting failure modes the system recovers.

By Dongxu Yang

llms agents benchmarks safety

Hugging Face Trending Papers

Jul 7

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable. We show that failure is predictable early from the agent's internal representations: lightweight per-round probes on hidden activations anticipate eventual episode failure as early as the first interaction round, where scorers reading only the agent's observable behavior are barely better than chance.

llms agents

arXiv AI

Sep 25

PROOF: Profiling Reliability of Object-Level Facts in Large Language Models

PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.

By Andrei Chetvergov, Mikhail Solovev, Timofei Sivoraksha, Stepan Ukolov, Valeriia Kuschenko, Alexander Evseev, Sergey Bolovtsov

llms benchmarks safety

arXiv AI

Sep 28

A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory

The paper introduces the Correlated Promotion Benchmark (CPB) to evaluate how agents decide whether to admit claims into shared memory, addressing the risk of repeating false claims. CPB offers two modes: CPB-Static, a frozen test set with fixed gold actions, and CPB-Live, which runs multi‑agent teams and tracks source lineage. Experiments across eight admission policies and four agent families show that deduplication reduces false claims but also discards true ones, while gating on declared source type most effectively limits false adoption.

By Xiaoyang Li, Yiqi Wang, Chencheng Zhu, KE XU, Wencheng Yang, Zequn Sun, Pingan Song, Yiqun Duan, Taotao Cai

agents benchmarks

arXiv AI

Jun 10

Deployment-Time Memorization in Foundation-Model Agents

arXiv:2606. 10062v1 Announce Type: new Abstract: Foundation-model agents are increasingly long-lived systems that remember users across interactions, making memorization an explicit deployment-time function rather than solely a property of model weights.

By Lei (Rachel), Chen, Guilin Zhang, Kai Zhao, Dalmo Cirne, Andy Olsen, Xu Chu, Zeke Miller, Alet Blanken, Amine Anoun, Jerry Ting

llms agents nlp safety

arXiv Machine Learning

Jul 13

Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.

By Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra, Matthias Hein

llms benchmarks safety

arXiv AI

Jun 10

Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models

arXiv:2606. 10949v1 Announce Type: new Abstract: Persistent memory systems promise to make LLMs more helpful by storing user beliefs over time.

By Shelly Bensal, Axel Magnuson, Aparna Balagopalan, Daniel M. Bikel

llms benchmarks

arXiv Machine Learning

Jun 25

Erased, but Not Gone: Output Forgetting Is Not True Forgetting

arXiv:2606. 25001v1 Announce Type: new Abstract: Machine unlearning (MU) is commonly judged by output forgetting, such as low forget-set accuracy or reduced logit-level membership inference.

By Teresa Pui Yee Yong, Win Kent Ong, Chee Seng Chan

Related stories