arXiv AI

Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agents

arXiv Computation and Language
Sep 16

Ask Now, Use Later: Benchmarking the Proactivity Gap in Long-Lived LLM Agents

The paper introduces ATRBench, a benchmark that measures the proactivity gap in long‑lived LLM agents by evaluating their ability to ask for user preferences that are not needed immediately but may be useful in future sessions. It defines the Ask‑to‑Remember (ATR) task, where agents must decide whether to request a reusable preference now, and shows that current state‑of‑the‑art agents perform significantly below an oracle. The study identifies preference acquisition as the main bottleneck and provides a diagnostic framework for improving agent proactivity.

By Bin Wu, Guanyun Zou, Bingbing Wang, Huan Zhao, Chuan Shi
Hugging Face Trending Papers
Jul 7

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable. We show that failure is predictable early from the agent's internal representations: lightweight per-round probes on hidden activations anticipate eventual episode failure as early as the first interaction round, where scorers reading only the agent's observable behavior are barely better than chance.

Hugging Face Trending Papers
Jul 1

MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators. However, memory is not always beneficial: retrieved memories often induce a critical issue of sycophancy, causing agents to over-align with the user at the cost of factual accuracy or objective reasoning.

Hugging Face Trending Papers
6d ago

Reliable Memory from Unstable Growth: Curating Self-Written Agent Playbooks with Randomized Experiments

The paper introduces Randomized Block Curation (RBC), a method for selecting useful lessons from language‑model agents’ persistent textual memory. RBC evaluates the impact of grouped lessons through randomized experiments on practice tasks, retaining only those that demonstrably improve task success. Experiments across two memory systems, three models, and three benchmarks show RBC raises average success to 66.9, outperforming both full playbooks and agent‑chosen lessons.