arXiv Computation and Language
Oct 16

BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

BASIS is a benchmark that tests large language model agents on their ability to infer users’ mental states, detect belief‑reality mismatches, and use this insight to craft better solutions in interactive settings. The study shows that current models often ignore users’ mental states, leading to poorer outcomes, while explicitly eliciting and incorporating such information improves solution quality. Additional experiments compare BASIS with Chain‑of‑Thought prompting, targeted training, and multi‑agent collaboration to demonstrate the practical benefits of mental‑state awareness.

By Minyuan Ruan, Ziyue Wang, Kaiming Liu, Yunghwei Lai, Peng Li, Yang Liu
arXiv Machine Learning
Oct 15

Narrowing the Collaboration Gap, Probably

arXiv:2610.19215v1 Announce Type: new Abstract: Large language models are increasingly deployed as teams of agents that hold different private information about a shared task. How should such agents...

By Mirah Shi, Marcel Hussing, Ira Globus-Harris, Natalie Collina, Aaron Roth, Surbhi Goel