Fig (Finding Interactions in Games) is an agentic system that performs targeted elicitation interventions on game environments to study multiagent behaviors. It takes a natural‑language behavior description, decomposes it into sub‑behaviors, generates or selects appropriate social games, and applies interventions to the game engine, opponents, instructions, and context. The system evaluates model behavior using mechanical checks and synthetic rubric judgments, and human review validates the interventions, generated games, and evaluation mechanisms. In experiments, models attempted the target behavior in 45.2% of episodes on average for six test behaviors, and Fig’s interventions increased attempt rates by 10.9% compared to matched control games.
arXiv:2511. 04500v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as decision-making agents in high-stakes domains and as imitators of human behavior in the social and behavioral sciences.
arXiv:2605.27068v2 Announce Type: replace-cross Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Mo...
arXiv:2606. 08200v1 Announce Type: new Abstract: Evaluating LLM-powered interactive social agents is challenging because socially relevant behaviors depend not only on isolated outputs, but also on prior interactions, social roles, and downstream actions.
The paper argues that artificial agentic systems, which operate as behavioral systems by interacting with dynamic environments, pursuing goals, and adapting over time, should be evaluated through systematic observation, perturbation, and interpretation of their actions rather than solely on performance outcomes. It draws on lessons from behavioral sciences to motivate this position and proposes a research agenda that includes methods for recovering decision strategies from action sequences, constructing environments that isolate behavioral differences, and probing emergent dynamics in multi‑agent systems. These directions aim to establish a rigorous science of AI behavior.
arXiv:2609.36308v1 Announce Type: new Abstract: Reinforcement learning has helped AI agents solve increasingly difficult tasks, but high rewards do not always reflect the work users intended. In rece...
BenchShield is a formal, model-backed instrumentation layer designed to protect reward integrity in large language model (LLM) agent benchmarks. It uses a finite lifecycle model of reward-relevant events to run a static, phase-aware taint analysis that flags potential reward-hacking paths before execution, and a runtime analysis that attributes concrete agent actions and provides evidence-backed claims. The system was evaluated on a corpus of 456 adjudicated trajectories from over 31,000 public agent runs across three benchmarks, showing significant improvements in recall, coverage, and cost efficiency compared to a baseline hackability scanner.
PROBE is an algorithm that trains language‑model auditors to probe deployed tool‑using agents for policy violations through realistic, multi‑turn, tool‑mediated dialogue. By framing auditing as a two‑player Markov game and using a composite reward that balances violation discovery, behavioral plausibility, conversation progress, and policy diversity, PROBE outperforms zero‑shot adversaries in uncovering broader and higher‑yield policy violations. The study compares GRPO, DPO, and GFlowNet optimizations, demonstrating that multi‑turn behavioral auditing is a distinct and necessary axis of agent evaluation, complementing existing capability and safety benchmarks.
arXiv:2608. 01193v1 Announce Type: cross Abstract: An AI development race creates a multi-agent safety dilemma.
Language-model agents act through repeated cycles of observation, reasoning, and action selection, making safety monitoring depend on both internal model state and environment context. We study reward-hacking monitors in ReAct-style agents acting in Gameable ALFWorld and WebShop.
arXiv:2607. 02975v1 Announce Type: new Abstract: Effective agency in social environments depends on when an agent seeks knowledge, when it acts, and whether its actions are justified by acquired information.
arXiv:2606. 06223v1 Announce Type: new Abstract: Language-model agents act through repeated cycles of observation, reasoning, and action selection, making safety monitoring depend on both internal model state and environment context.
arXiv:2606. 04923v1 Announce Type: cross Abstract: Rubric-based reinforcement learning (RL) uses an LLM-as-a-Judge (LaaJ) to score model outputs according to rubrics as rewards.