arXiv AI
Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 5
Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning
arXiv:2610.02715v1 Announce Type: new Abstract: Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses its...
arXiv Computer Vision
Oct 2
Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation
Ego2Act is a new benchmark that tests video generation models on goal‑directed, egocentric manipulation tasks. It contains 2,640 videos from 110 real‑world tasks, each requiring multiple steps of object manipulation to achieve a high‑level goal. The benchmark includes Ego2ActJudge, a reference‑free evaluation pipeline that better aligns with human judgments of task completion and physics plausibility.
arXiv AI
4d ago
Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction
The paper introduces Multi-agent Egocentric World Model (ME-World), a framework that generates synchronized first-person video streams for multiple agents interacting in a shared environment. It addresses fine-grained embodied actions by ensuring cross-view action consistency, shared-environment consistency, and consistent state updates across agents. Experiments on real and synthetic data demonstrate that ME-World outperforms existing methods in shared-world consistency, action control, identity preservation, and video quality.
arXiv AI
Jun 29
EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning
arXiv:2603. 09731v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly considered as a foundation for embodied agents, yet it remains unclear whether they can reliably reason about the long-term physical consequences of actions from an egocentric viewpoint.
arXiv Computer Vision
Aug 25
WorldMind: Decoupled Game World Model for State-Aware NPC Behavior
WorldMind is a decoupled framework for state-aware NPC behavior in game world models, separating interactive world modeling into four layers: Understanding, Decision, Control, and Generation. It constructs a compact state from generated frames, reasons over it to plan NPC actions, translates actions into temporally aligned conditions, and synthesizes visual outcomes. Experiments on the newly introduced BOSS-140K dataset show that WorldMind achieves more tactically appropriate and coherent NPC behavior than baseline models in about 70% of pairwise comparisons.
arXiv AI
Jul 2
EgoSim: Egocentric World Simulator for Embodied Interaction Generation
arXiv:2604. 01001v2 Announce Type: replace-cross Abstract: We introduce EgoSim, a closed-loop egocentric world simulator that generates spatially consistent interaction videos and persistently updates the underlying 3D scene state for continuous simulation.