Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inference from a partially observed multimodal episode to...
arXiv:2609.17064v1 Announce Type: new Abstract: Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inferenc...
arXiv:2608. 10765v1 Announce Type: new Abstract: Recognizing human behavior across levels of abstraction, from atomic actions to long-horizon intentions, requires data annotated along a semantic hierarchy.
arXiv:2608. 11739v1 Announce Type: cross Abstract: The prevailing recipe for Vision-Language-Action (VLA) models couples a pretrained VLM with a separately trained flow-matching action expert.
OmniBrain-0 is a lightweight, unified embodied agent that coordinates manipulation and navigation through a shared high‑level decision model. It integrates task understanding, planning, progress tracking, and feedback‑driven recovery, and is trained via embodied pretraining, domain‑specialized agent distillation, and multi‑teacher on‑policy distillation. The framework also introduces Rubric‑Guided Recursive Self‑Improvement, a data flywheel that refines rubrics and trajectories for continual domain‑expert updates, enabling efficient transfer and consolidation of embodied capabilities across multiple benchmarks and real‑world tasks.
The paper introduces Intention Distillation (INDI), a method that injects behavior-level intent into Vision‑Language‑Action (VLA) model decoders by leveraging a frozen teacher vision‑language model to interpret demonstrations. During training, the teacher processes the current observation, instruction, coarse action summary, and execution video, producing a multimodal intent representation that the VLA decoder uses alongside trajectory and execution features to predict actions. Experiments on SimplerEnv‑Bridge, RoboCasa Kitchen, and real‑world tasks show that INDI consistently improves success rates, especially on longer‑horizon tasks, demonstrating that explicit modeling of semantic intent benefits action decoders.
Uni-LaDiR (Unified Latent Diffusion Reasoner) is a new framework that unifies multimodal reasoning by mapping teacher reasoning steps from different modalities into a shared latent space of thought tokens. It employs a diffusion model to predict the next block of thought tokens, jointly training the encoder and reasoner with shared weights to ensure tokens are both useful and predictable. The approach achieves relative gains of 7.3% on visual reasoning benchmarks and 6.1% on robot manipulation tasks compared to the strongest baselines.
Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was dem...
Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.
The paper introduces ProAction, a multimodal dataset of 10,000 samples comprising visual, audio, and text inputs across 12 daily-life scenarios, designed to support the Proactive Robot Action Reasoning (ProRobo) problem. It presents a two-stage human-in-the-loop annotation pipeline that incorporates appraisal and Theory-of-Mind considerations to generate cognitively grounded high-level action labels. The authors benchmark multimodal large language models and propose MMC2Act, showing that training on ProAction significantly improves proactive action reasoning compared to general-purpose models.
arXiv:2607. 14149v1 Announce Type: new Abstract: Although large language models (LLMs) have set benchmarks for zero-shot reasoning, their deployment remains cost-prohibitive and environmentally taxing.
arXiv:2510. 14828v3 Announce Type: replace Abstract: Improving the reasoning capabilities of embodied agents is crucial for robots to complete complex human instructions in long-view manipulation tasks successfully.