arXiv AI
Aug 17
When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict
arXiv:2608. 13921v1 Announce Type: new Abstract: LLM agents increasingly maintain personal memory across sessions, but it can conflict.
Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
The paper introduces a new evaluation for large language model agents called epistemic humility (EH), which measures an agent’s ability to recognize, act on, and communicate uncertainty when evidence conflicts with prior beliefs. EH is operationalized through three trajectory-level behaviors—Identify, Solve, and Escalate (ISE)—and tested in both controlled and naturally occurring knowledge conflict scenarios. Results show that higher task accuracy does not guarantee greater epistemic humility, and that model-level interventions can improve EH but often reduce accuracy, indicating that humility depends on the interplay between the backbone model, agent harness, and environment.
Related stories
Hugging Face Trending Papers
Aug 14
When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict
LLM agents increasingly maintain personal memory across sessions, but it can conflict. Preferences depend on context, behavior evolves, and sources can conflict. When a query lacks context, time, or s...
arXiv AI
Aug 17
Demystifying Agent Skills: Why They Work-Until They Don't
arXiv:2608. 14036v1 Announce Type: new Abstract: Skills have emerged as a practical and effective approach for enhancing LLM agents at inference time through structured packages of knowledge.
arXiv AI
Aug 17
Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent Messages
arXiv:2608. 14375v1 Announce Type: new Abstract: Multi-agent reasoning systems often use agreement, confidence, or automated scores to decide which messages should shape a final answer.
arXiv AI
Jul 28
Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure
arXiv:2603. 28371v2 Announce Type: replace-cross Abstract: When an agent can articulate why something works, we typically take this as evidence of genuine understanding.
arXiv AI
Aug 25
Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models
arXiv:2608.21377v1 Announce Type: cross Abstract: Sycophancy in large language models, the tendency to prioritize user agreement over truthful responses, has been documented extensively but studied p...
arXiv AI
Jul 7
Ask the World Before Acting: Environment Probing for Calibrated Agent World Models
arXiv:2606. 31422v2 Announce Type: replace Abstract: Language agents acting over long horizons must maintain beliefs about tool states, object locations, graph edges, and subgoal dependencies.
arXiv AI
Jul 20
Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
arXiv:2607. 15434v1 Announce Type: cross Abstract: Multi-agent systems routinely place one AI agent in authority over another.
arXiv AI
Jul 21
Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict
arXiv:2605. 14473v4 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) is usually evaluated by whether the final answer is correct.
arXiv AI
Jul 16
When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents
arXiv:2602. 11619v2 Announce Type: replace Abstract: Running the same LLM agent on identical inputs yields 2.
arXiv AI
Oct 1
Beyond the Shadows of Plato's Cave: Evaluating False Memory in Autonomous Agents via Counterfactual Reasoning
The paper introduces FAME, a training‑free framework that evaluates false memory in autonomous agents by tracking how their internal beliefs shift under counterfactual scenarios. False memory, defined as biases arising from spurious correlations, environment shifts, or knowledge conflicts, is hard to detect with standard methods. FAME measures concept drift in hidden states, achieving AUROCs between 76.2% and 96.7% and outperforming baselines on benchmarks such as GSM‑Symbolic, GitChameleon, and BigBench‑Hard.
arXiv AI
Jul 15
Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents
arXiv:2607. 12267v1 Announce Type: cross Abstract: Language agents that interleave reasoning and tool use degrade sharply as reasoning chains lengthen, even when each individual step is easy.