arXiv Computation and Language
Oct 16

BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

BASIS is a benchmark that tests large language model agents on their ability to infer users’ mental states, detect belief‑reality mismatches, and use this insight to craft better solutions in interactive settings. The study shows that current models often ignore users’ mental states, leading to poorer outcomes, while explicitly eliciting and incorporating such information improves solution quality. Additional experiments compare BASIS with Chain‑of‑Thought prompting, targeted training, and multi‑agent collaboration to demonstrate the practical benefits of mental‑state awareness.

By Minyuan Ruan, Ziyue Wang, Kaiming Liu, Yunghwei Lai, Peng Li, Yang Liu
Hugging Face Trending Papers
Jun 30

Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action

Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM).