Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
Read the original on arXiv AI →The paper introduces a new evaluation for large language model agents called epistemic humility (EH), which measures an agent’s ability to recognize, act on, and communicate uncertainty when evidence conflicts with prior beliefs. EH is operationalized through three trajectory-level behaviors—Identify, Solve, and Escalate (ISE)—and tested in both controlled and naturally occurring knowledge conflict scenarios. Results show that higher task accuracy does not guarantee greater epistemic humility, and that model-level interventions can improve EH but often reduce accuracy, indicating that humility depends on the interplay between the backbone model, agent harness, and environment.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.