Hugging Face Trending Papers

Jev-ToM: Evaluating Theory of Mind in Typed Decision Models

Read the original on Hugging Face Trending Papers →

Jev-ToM evaluates Theory of Mind (ToM) in typed decision models, comparing five such models and seven generative baselines on 11,117 English text questions from nine ToM benchmarks across six capability groups. The TypeSafe Jev model achieves 73.2% source‑balanced accuracy, ranking first or second in all six capabilities, including leading beliefs, goals/desires/intentions, and emotions. Other typed models show distinct strengths, and resource comparisons suggest Jev could reduce cost and waiting time for repeated ToM selection, while confidence‑based selection and candidate expansion reveal task‑dependent challenges and opportunities for improvement.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
6d ago

Jev-ToM: Evaluating Theory of Mind in Typed Decision Models

arXiv:2610.18610v1 Announce Type: cross Abstract: Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternativ...

By Kawai Chung, Chunkit Chan, Yuxuan Liu, Yauwai Yim, Xi Yang, Wenbin Hu, Huihao Jing, Chiu Yeung Tang, Changxuan Fan, Ki Sen Hung, Tsun On Kwok, Haoran Li
arXiv Machine Learning
Sep 15

Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States

arXiv:2609.15972v1 Announce Type: cross Abstract: As language models become more capable, long-term collaboration in learning, reasoning, and decision-making calls for a deeper understanding of the p...

By Zixuan Wang, Yufan Zhou, Jinzhou Tang, Xinle Yu, Chengjun Wu, Lyumanshan Ye, Zhaoxiang Feng, Letian Peng, Adyasha Patra, Fan Bai, Enze Ma, Zhengding Hu, Jianyang Gu, Zhao Wang, Yufei Ding, Jingbo Shang, Tianmin Shu, Zhiting Hu, Zhen Wang
arXiv AI
Aug 25

DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

DialToM is a Theory of Mind benchmark created from naturalistic human-human dialogues, using a multiple-choice format. It introduces a State-Driven Diagnostic Probe that requires models to predict dialogue trajectories based solely on isolated mental-state profiles, without dialogue context. The evaluation shows that large language models are good at inferring mental states (Literal ToM) but struggle to use them for social forecasting (Functional ToM), while a domain expert scores 100% accuracy, highlighting a clear human‑AI gap.

By Neemesh Yadav, Palakorn Achananuparp, Jing Jiang, Ee-Peng Lim
arXiv Computation and Language
5d ago

BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

BASIS is a benchmark that tests large language model agents on their ability to infer users’ mental states, detect belief‑reality mismatches, and use this insight to craft better solutions in interactive settings. The study shows that current models often ignore users’ mental states, leading to poorer outcomes, while explicitly eliciting and incorporating such information improves solution quality. Additional experiments compare BASIS with Chain‑of‑Thought prompting, targeted training, and multi‑agent collaboration to demonstrate the practical benefits of mental‑state awareness.

By Minyuan Ruan, Ziyue Wang, Kaiming Liu, Yunghwei Lai, Peng Li, Yang Liu