← Back to all news
arXiv AI By Rodrigo Reveron-Fonseca

Memory or a Clock? Auditing a Memory Advantage in XLand-MiniGrid with Seeds, Maps and Evaluation Noise Apart

Read the original on arXiv AI →

The paper investigates why an LSTM agent outperforms a memory‑free MLP on a partially observable XLand‑MiniGrid task. By separating training seed, map, and evaluation randomness across 60 runs, the authors find that much of the 25‑percentage‑point gap is due to noise, and that providing the time step to a memory‑free MLP reduces the gap to a few points. A linear probe shows the LSTM’s state largely encodes the time step, indicating that the apparent memory advantage is largely an advantage of knowing episode time. "whyItMatters":"The study clarifies that the perceived memory benefit in this benchmark is largely attributable to time‑step awareness rather than true memory, guiding future baseline design and evaluation practices."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 4

Learning What Not to Forget: Long-Horizon Agent Memory from a Few Kilobytes of Learning

The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.

By Nusrat Jahan Lia, Aritra Mazumder
llmsagentsefficiency
More like this →
arXiv AI
Sep 3

The Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory Agents

The paper investigates how persistent memory in AI agents can lead to over‑trust in stale facts, creating a "Memory Trust Gap" that worsens as model capability increases. Using a benchmark with Benefit and Safety suites across Qwen3 models of varying sizes, the authors show that larger models are more prone to harmful over‑trust, especially when metadata is absent or misleading. They also demonstrate that mitigation strategies such as exposing metadata or pre‑resolving conflicts improve accuracy, but the effectiveness depends on model size and dataset.

By Jundong Hu, Shekar Ramachandran
llmsagentsbenchmarkssafety
More like this →