Belief Without Behavior: Measuring the Translation of Theory of Mind into Coordinated Social Action in Vision-Language Models

The paper introduces MOSAIC, a benchmark that tests how vision‑language models translate theory‑of‑mind (ToM) reasoning into coordinated social actions across verbal and nonverbal channels. Across 200 trials per model, 13 models—including 11 VLMs—failed to align their behaviors with ToM constraints, revealing bottlenecks in generating coherent nonverbal signals and interpreting others’ actions. A structured model with an explicit ToM module, PCM‑LLM, succeeded in all conditions, indicating that belief‑action coupling is crucial for these tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• robotics
• multimodal
• benchmarks