ProAct is a dual‑system framework for real‑time embodied social interaction that separates a low‑latency Behavioral System, which streams multimodal interaction and generates continuous non‑verbal motion, from a slower Cognitive System that performs long‑horizon social reasoning and produces proactive intentions. The Cognitive System uses an efficient memory mechanism and a user‑motivation prediction module to decide when to intervene, while the Behavioral System translates these intentions into fluid motion via an intention‑conditioned streaming flow‑matching generator with a disentangled ControlNet branch. The framework is deployed on a physical humanoid robot and validated through real‑world user studies, motion‑generation benchmarks, and a new ProActBench benchmark for proactive trigger detection and restraint.
arXiv:2610.16967v1 Announce Type: cross Abstract: Physical AI is commonly formulated around an explicitly specified task: given a goal or instruction, a system perceives, reasons, and acts. In many r...
The paper introduces ProAction, a multimodal dataset of 10,000 samples comprising visual, audio, and text inputs across 12 daily-life scenarios, designed to support the Proactive Robot Action Reasoning (ProRobo) problem. It presents a two-stage human-in-the-loop annotation pipeline that incorporates appraisal and Theory-of-Mind considerations to generate cognitively grounded high-level action labels. The authors benchmark multimodal large language models and propose MMC2Act, showing that training on ProAction significantly improves proactive action reasoning compared to general-purpose models.
arXiv:2606. 14778v1 Announce Type: cross Abstract: Long-term action anticipation (LTA) aims to predict an ordered sequence of future verb-noun actions from a partially observed video.
StreamPI introduces a streaming multimodal temporal modeling framework that enhances Vision‑Language‑Action models by adding temporal reasoning without extra parameters. It anchors each visual observation and language instruction pair as a temporal unit, using bidirectional attention for cross‑modal fusion and causal attention for autoregressive streaming inference. The method employs random‑interval streaming training to improve robustness and leverages the LLM backbone’s length extrapolation to inherit pretrained weights, achieving superior performance over pi0.5 on real‑robot and simulation tasks.
Vision-Language-Action (VLA) models have demonstrated effectiveness in robot manipulation, yet state-of-the-art models such as pi0.5 operate under a single-frame paradigm, limiting their ability to re...
Long-term physical coexistence with intelligent robots requires more than capable robot policies. A persistent robotic assistant must support diverse user-facing interfaces, maintain long-horizon memory of people and preferences, coordinate across robot embodiments, and translate human intent into safe physical execution.
arXiv:2607. 02542v1 Announce Type: new Abstract: General-purpose embodied agents must understand multimodal instructions, anticipate how their environment will evolve, and produce precise control actions over extended horizons.
arXiv:2608.22971v2 Announce Type: replace Abstract: Embodied AI has emerged as a key paradigm for autonomous systems operating in complex physical environments. While end-to-end models have establish...
arXiv:2606. 31167v1 Announce Type: cross Abstract: VLA models have emerged as a powerful paradigm for transferring semantic knowledge from web-scale data to physical robotic control.
The paper introduces Movens-5M, a dataset of 15 million human‑motion sequences and 5 million humanoid‑executed motions derived from raw Internet videos, aligned with text, RGB video, speech, and music. Using this data, the authors present Movens‑Zero, a scalable architecture that jointly learns across modalities and supports real‑time closed‑loop execution, enabling motion generation from text, speech, music, and video. They also release Movens‑Intent, a benchmark of 8,192 expert‑verified condition‑motion pairs, and demonstrate that scaling data quality and model capacity improves performance across all modalities.
arXiv:2607. 03093v1 Announce Type: cross Abstract: Thinking has emerged as a critical capability for Large Language Models (LLMs) tackling complex tasks.
Realtime-Bench is a new benchmark that evaluates whether frontier models can handle real‑time interactive tasks where delayed actions directly impact success. The suite includes dynamic tasks that advance under different time‑clock regimes—Wall Clock, Paused Clock, and Token Clock—to isolate serving latency from a model’s temporal reasoning ability. Results show that even the best models achieve only 44% success under real‑time constraints, highlighting the challenge of balancing decision quality with timely execution.