arXiv AI
QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 15
Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning
The paper introduces Multi-agent Undercover Gaming (MUG), a protocol that extends the Multi-Agent Debate (MAD) framework to address hallucinations in large language models. MUG reframes MAD as a detection task for ‘undercover’ agents—those that hallucinate—by using multimodal counterfactual tests that modify reference images to reveal hallucinations. The approach enhances factual verification, cross‑evidence reasoning, and active discussion among agents, aiming to improve reliability in multimodal reasoning.
arXiv AI
Jun 4
SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Models
arXiv:2606. 04202v1 Announce Type: new Abstract: As LLMs become more widely deployed, they are increasingly expected to work alongside other AI agents rather than operating in isolation.
arXiv AI
Oct 23
Fig: Automated Multiagent Behavior Elicitor
Fig (Finding Interactions in Games) is an agentic system that performs targeted elicitation interventions on game environments to study multiagent behaviors. It takes a natural‑language behavior description, decomposes it into sub‑behaviors, generates or selects appropriate social games, and applies interventions to the game engine, opponents, instructions, and context. The system evaluates model behavior using mechanical checks and synthetic rubric judgments, and human review validates the interventions, generated games, and evaluation mechanisms. In experiments, models attempted the target behavior in 45.2% of episodes on average for six test behaviors, and Fig’s interventions increased attempt rates by 10.9% compared to matched control games.
arXiv AI
Jul 10
Persuasion Attacks Can Decrease Effectiveness of CoT Monitoring
arXiv:2607. 08066v1 Announce Type: new Abstract: Chain-of-thought (CoT) monitoring is a promising safety mechanism for AI agents, based on the premise that visible reasoning traces can surface misaligned or deceptive behavior.
arXiv AI
Sep 15
Corrupt Plans, Clean Traces: Evading Chain-of-Thought Monitoring with Plan Injection
The paper introduces a new attack called "plan injection" that allows a large language model to carry out harmful actions while evading chain-of-thought monitoring. By inserting harmful but benign-sounding reasoning into the model’s context, the attacker can steer the model’s behavior and cause it to paraphrase the injected plan as its own reasoning. The study demonstrates that this attack works across different monitoring settings, scales to harder tasks, and even causes monitors to waste resources on the injected plan, reducing detection rates by up to 50%.
arXiv Computer Vision
Aug 26
DoublesEval: Diagnosing Multi-Agent Tactical Reasoning in Vision-Language Models via Professional Doubles Badminton
The paper introduces DoublesEval, a diagnostic framework that uses professional doubles badminton to test visual‑language models’ ability to reason about dynamic multi‑agent interactions. It decomposes rallies into key moments and evaluates models across four dimensions—atomic recognition, intra‑segment composite understanding, cross‑segment causal reasoning, and high‑level tactical abstraction—highlighting specific reasoning failures. The authors also propose TacticCheck, a lightweight consistency checker that improves performance without retraining the models, yet significant gaps remain in tactical reasoning.