arXiv AI
Beyond Successful Play: Diagnosing Opponent Modeling in Large Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 11
Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics
arXiv:2608. 09638v1 Announce Type: new Abstract: Theory of Mind (ToM) is essential for agent interactions, yet existing evaluations either rely on static scenarios that oversimplify mental-state reasoning or interactive settings that provide limited diagnostic insight.
arXiv AI
Sep 1
When Should Models Change Their Minds? Contextual Belief Management in Large Language Models
arXiv:2605.30219v2 Announce Type: replace Abstract: Long-horizon interactions require language models to manage accumulating information: when to update their state, when to preserve their state, and...
arXiv Computation and Language
4d ago
BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions
BASIS is a benchmark that tests large language model agents on their ability to infer users’ mental states, detect belief‑reality mismatches, and use this insight to craft better solutions in interactive settings. The study shows that current models often ignore users’ mental states, leading to poorer outcomes, while explicitly eliciting and incorporating such information improves solution quality. Additional experiments compare BASIS with Chain‑of‑Thought prompting, targeted training, and multi‑agent collaboration to demonstrate the practical benefits of mental‑state awareness.
arXiv AI
Aug 28
Assessing mentalization in humans and large language models
The study evaluates mentalization—the capacity to infer others’ beliefs and intentions—in large language models (LLMs) using two economic games and cognitive computational modeling. Researchers tested 2,099 LLM agents from four model families (DeepSeek, GPT‑4.1, GPT‑5, Gemini 2.0 Flash) against opponents of varying sophistication, comparing their performance to 251 human participants. Results show that LLMs exhibit distinct mentalizing behaviors that vary by model provider and size, with strategic prompting generally enhancing performance; notably, GPT‑5 agents adapt their recursive reasoning depth to match opponent sophistication, outperforming humans in one task.
arXiv AI
Aug 18
Beyond Asking: A Pipeline for Personalized Game Generation that Reads Players from Behavior
arXiv:2608. 16196v1 Announce Type: new Abstract: Personalized game generation requires inferring a player's abilities and behavioral style from how they play.
arXiv AI
Oct 9
Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Memento 3 is a frozen large‑language‑model agent that learns explicit world models by maintaining a natural‑language rulebook in external memory. The agent iteratively observes, reflects, revises the rulebook, compiles it into executable code, and verifies updates through prediction error and replay, accepting only code that the LLM deems faithful to the rulebook. In experiments, a single‑model version solved all 25 public ARC‑AGI‑3 games with perfect efficiency, and a learned feedback controller achieved perfect scores in Atari Pong without further LLM calls.