StreamPI introduces a streaming multimodal temporal modeling framework that enhances Vision‑Language‑Action models by adding temporal reasoning without extra parameters. It anchors each visual observation and language instruction pair as a temporal unit, using bidirectional attention for cross‑modal fusion and causal attention for autoregressive streaming inference. The method employs random‑interval streaming training to improve robustness and leverages the LLM backbone’s length extrapolation to inherit pretrained weights, achieving superior performance over pi0.5 on real‑robot and simulation tasks.
By Zhe Liu, Jinghua Hou, Yuxiang Lu, Zhenya Yang, Xianzhe Fan, Junwei Luo, Junyi Li, Ruihua Han, Zhi Hou, Hengshuang Zhao
arXiv:2610.00981v1 Announce Type: cross
Abstract: We focus on language-conditioned flow-based manipulation, where robot flows (robot velocity fields) serve as embodiment-agnostic, motion-centric repr...
By Shota Kobayashi, Koki Seno, Daichi Yashima, Komei Sugiura
LookStep is a new end‑to‑end framework for Vision‑Language Navigation that integrates Language‑Centric Future State Modeling with an Event‑Driven Rolling Memory. It uses language labels to predict coarse navigation progress and future states for candidate actions, and autonomously decides which observations to store in a bounded memory with semantic roles. Empirical results show that LookStep outperforms existing methods on VLN‑CE tasks, achieving a 49.7% success rate on R2R‑CE Val‑Unseen while improving memory efficiency and reducing data requirements.
By Kun-Yang Yu, Yingzhe Li, Hongyu Xu, Shi-Yu Tian, Zhi Zhou, Yang Chen, Ming Yang, Sheng Wang, Qing Yu, Lan-Zhe Guo, Yu-Feng Li
arXiv:2608. 04765v1 Announce Type: cross Abstract: Vision-language-action (VLA) models provide a unified paradigm for connecting visual perception, language understanding, and robotic control.
By Houze Xu, Jizhong Li, Ziyi Ye
Vision-Language-Action (VLA) models have demonstrated effectiveness in robot manipulation, yet state-of-the-art models such as pi0.5 operate under a single-frame paradigm, limiting their ability to re...
arXiv:2606. 12550v1 Announce Type: cross Abstract: Open-world mapless navigation from sparse language instructions requires resolving underspecified goals and inferring which environmental cues are relevant for reaching the goal.
By Arthur Zhang, Carl Qi, Donne Su, Xiangyun Meng, Amy Zhang, Joydeep Biswas