arXiv Computation and Language

From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks

arXiv Computation and Language
Oct 16

BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

BASIS is a benchmark that tests large language model agents on their ability to infer users’ mental states, detect belief‑reality mismatches, and use this insight to craft better solutions in interactive settings. The study shows that current models often ignore users’ mental states, leading to poorer outcomes, while explicitly eliciting and incorporating such information improves solution quality. Additional experiments compare BASIS with Chain‑of‑Thought prompting, targeted training, and multi‑agent collaboration to demonstrate the practical benefits of mental‑state awareness.

By Minyuan Ruan, Ziyue Wang, Kaiming Liu, Yunghwei Lai, Peng Li, Yang Liu
arXiv Machine Learning
Oct 15

Narrowing the Collaboration Gap, Probably

arXiv:2610.19215v1 Announce Type: new Abstract: Large language models are increasingly deployed as teams of agents that hold different private information about a shared task. How should such agents...

By Mirah Shi, Marcel Hussing, Ira Globus-Harris, Natalie Collina, Aaron Roth, Surbhi Goel
Hugging Face Trending Papers
Jun 30

Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action

Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM).

arXiv Computation and Language
Oct 15

What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds

The paper introduces HICO-Bench, a benchmark of 180 everyday coordination scenarios where two agents with private information converse and then record a shared plan. A program verifies that the plans are feasible and an LLM judge checks that the conversation supports the plan, defining a Verified Success. Using Paired-Credit Policy Optimization (PaCPO), the authors improve the Verified Success rate of Qwen3-8B from 20.4% to 28.3%.

By Caixin Kang, Ke Wang, Yuchuan Wu, Xiaoqiang Wang, Ling Li, Bingxue Qiu, Yongbin Li, Yoichi Sato
arXiv AI
Oct 28

Who Knows What - and What Does It Support? Perspective-Bounded Evidence Calibration for Multimodal Theory of Mind

The paper introduces PIP-ToM, a framework that addresses the evidence-admissibility problem in multimodal Theory of Mind (ToM) by ensuring that only evidence accessible to an agent and diagnostic of its mental state is used in hypothesis evaluation. PIP-ToM builds agent‑specific evidence stores, aligns them with structured mental‑state claims, and calibrates Bayesian inference scores while preserving the underlying inference process. Using GPT‑4o, the method outperforms existing ToM approaches on MuMA‑ToM, MMToM‑QA, and Hi‑ToM benchmarks, achieving accuracies of 92.56%, 95.00%, and 95.00% respectively.

By Ziqi Liao, Jiaqi Shen, Yiming Na, Kedi Lyu, Shao-Yuan Lo, Hongxia Xie