Fig (Finding Interactions in Games) is an agentic system that performs targeted elicitation interventions on game environments to study multiagent behaviors. It takes a natural‑language behavior description, decomposes it into sub‑behaviors, generates or selects appropriate social games, and applies interventions to the game engine, opponents, instructions, and context. The system evaluates model behavior using mechanical checks and synthetic rubric judgments, and human review validates the interventions, generated games, and evaluation mechanisms. In experiments, models attempted the target behavior in 45.2% of episodes on average for six test behaviors, and Fig’s interventions increased attempt rates by 10.9% compared to matched control games.
By Anrui Gu, Zhichao Wang, Nicolai Ouporov
The article "Defining AI Agents: A Compendium of Criteria, Metrics, and Benchmarks" surveys the lack of a standard definition for AI agents and organizes this ambiguity into five dimensions: environmental interaction, learning and adaptation, autonomy, goal‑directed behavior, and temporal coherence. It reviews how each dimension has been conceptualized in prior work and compiles the metrics, benchmarks, and evaluation frameworks used to assess them. The authors also introduce the Agent Compendium, a public digital resource that extends these evaluation methods, aiming to provide a common structure for evaluating and comparing agent capabilities across AI systems.
By Mia Lassiter, Brinnae Bent
As AI agents are increasingly deployed in complex environments, understanding their behaviors becomes critical. Yet behavioral scientific research on AI agents remains manual and labor-intensive.
arXiv:2608. 10030v1 Announce Type: new Abstract: As AI agents are increasingly deployed in complex environments, understanding their behaviors becomes critical.
By Soo Yong Lee, Jongha Lee, Jaewan Chun, Hyunjin Hwang, Fanchen Bu, Ziv Ben-Zion, Taekwan Kim, Denny Borsboom, Jaemin Yoo, Kijung Shin
AI agents are commonly evaluated using task success, reward, latency, and cost. These metrics are useful, but they often miss important aspects of agent behavior: whether an agent explores too much, repeats itself too rigidly, uses tools effectively, reduces uncertainty over time, or remains robust across repeated runs.
arXiv:2602. 06841v4 Announce Type: replace Abstract: Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure.
By Sindhuja Chaduvula, Jessee Ho, Kina Kim, Aravind Narayanan, Ahmed Y. Radwan, Mahshid Alinoori, Muskan Garg, Dhanesh Ramachandram, Shaina Raza