The paper introduces Predicting Intention of Partner (PIP), a method for zero‑shot coordination in partially observable settings. PIP employs a Joint‑view VAE to create a partner representation from local observations and uses Partner‑State Belief networks to infer hidden partner states from interaction history. Experiments on Burrito‑PO, Overcooked‑PO, and Melting Pot, along with human evaluation, show that PIP outperforms other methods and demonstrates effective coordination with unseen partners under occlusion.
arXiv:2610.16895v1 Announce Type: new Abstract: LLMs are used as high-level controllers for embodied agents, but it remains unclear whether they support collaborative reasoning, such as anticipating...
arXiv:2609.39727v1 Announce Type: new Abstract: Cooperative language-model agents must coordinate over long horizons and adapt to changing environments and to partners with unfamiliar conventions, ye...
arXiv:2606. 14778v1 Announce Type: cross Abstract: Long-term action anticipation (LTA) aims to predict an ordered sequence of future verb-noun actions from a partially observed video.
The paper introduces Multi-agent Egocentric World Model (ME-World), a framework that generates synchronized first-person video streams for multiple agents interacting in a shared environment. It addresses fine-grained embodied actions by ensuring cross-view action consistency, shared-environment consistency, and consistent state updates across agents. Experiments on real and synthetic data demonstrate that ME-World outperforms existing methods in shared-world consistency, action control, identity preservation, and video quality.
arXiv:2605. 12920v3 Announce Type: replace-cross Abstract: Effective collaboration between embodied agents requires more than acting in a shared environment; it demands communication grounded in each agent's evolving understanding of the world.
The paper introduces MOSAIC, a benchmark that tests how vision‑language models translate theory‑of‑mind (ToM) reasoning into coordinated social actions across verbal and nonverbal channels. Across 200 trials per model, 13 models—including 11 VLMs—failed to align their behaviors with ToM constraints, revealing bottlenecks in generating coherent nonverbal signals and interpreting others’ actions. A structured model with an explicit ToM module, PCM‑LLM, succeeded in all conditions, indicating that belief‑action coupling is crucial for these tasks.
The paper introduces Watch, Infer, Coordinate, a benchmark and inference method for robots to deduce a partner’s physical constraints by observing joint behavior in physically coupled manipulation tasks. It demonstrates that understanding these constraints enables zero‑shot coordination on new tasks, achieving performance close to an oracle with true constraint knowledge. The study focuses on scenarios where a robot’s limitations may be unknown to its partner, such as hardware degradation or actuator faults.
arXiv:2608. 05673v1 Announce Type: new Abstract: Trajectory prediction has shifted toward structured formulations with explicit social modeling.
BOLT is a lightweight plug‑and‑play module that enables preparation‑free heterogeneous cooperative perception by adapting neighboring features online through ego‑as‑teacher distillation. It requires only ego predictions, no ground‑truth labels, and uses high‑confidence ego features to align cross‑agent feature domains while allowing neighbors to contribute in low‑confidence regions. With just 0.9 M trainable parameters, BOLT boosts AP@50 by up to 32.3 points over unadapted fusion and consistently outperforms ego‑only results on DAIR‑V2X and OPV2V.
Attacca is a new approach for training visual goal‑conditioned policies that can handle continuous long‑horizon task execution in embodied agents. It decouples goal images from the execution environment by sampling class‑compatible masked goals from other worlds, enabling the policy to ground targets in the current view via a target‑mask prediction head. The method also introduces behavioral‑phase conditioning to distinguish Search, Approach, and Interact stages, and achieves significant performance gains on Minecraft tasks, outperforming baselines by up to 7× on long‑horizon scenarios.
arXiv:2605.13335v2 Announce Type: replace Abstract: Embodied agents in household environments must plan under partial observation: they need to remember objects, track state changes, and recover when...
arXiv:2603.03768v2 Announce Type: replace-cross Abstract: Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller cha...