Robotics and embodied AI

Manipulation, locomotion, sim-to-real transfer and autonomous driving: learning systems that have to survive physics.

2,234 stories · RSS feed

arXiv AI

Jun 2

Improving Diffusion Planners by Self-Supervised Action Gating with Energies

arXiv:2603. 02650v2 Announce Type: replace-cross Abstract: Diffusion planners are a strong approach for offline reinforcement learning, but they can fail when value-guided selection favours trajectories that score well yet are locally inconsistent with the environment dynamics, resulting in brittle execution.

By Yuan Lu, Dongqi Han, Yansen Wang, Dongsheng Li

rag diffusion reinforcement-learning robotics benchmarks

arXiv AI

Jun 2

Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation

arXiv:2606. 00095v1 Announce Type: cross Abstract: Vision-Language Navigation (VLN) enables embodied agents to reach target locations in unseen environments by following language instructions.

By Kailing Li, Tianwen Qian, Lijin Yang, Yuqian Fu, Jingyu Gong, Xiaoling Wang, Liang He

llms agents robotics multimodal benchmarks

arXiv AI

Jun 2

AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

arXiv:2605. 29488v2 Announce Type: replace-cross Abstract: Conditional human motion generation remains a fundamental challenge in computer vision and robotics.

By Yiheng Li, Zhuo Li, Ruibing Hou, Yingjie Chen, Hong Chang, Hao Liu, Shiguang Shan

llms computer-vision nlp robotics multimodal

arXiv AI

Jun 2

VDSB-GWSyn: Diffusion Schr\"{o}dinger Bridge for Controllable and Anatomically Feasible Guidewire Synthesis in Coronary Angiography

arXiv:2606. 00109v1 Announce Type: cross Abstract: Coronary guidewire endpoint localization is a fundamental capability for computer-assisted PCI, and its importance increases as robot-assisted PCI is progressively adopted to reduce operator radiation exposure.

By Haoyuan Tang, Zhuo Zhang, Jialin Li, Shuai Xiao, Jiachen Yang

diffusion computer-vision robotics fine-tuning

arXiv AI

Jun 2

StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement

arXiv:2606. 00267v1 Announce Type: cross Abstract: Video world models (WMs) have shown promise for policy evaluation and improvement by imagining realistic future observations conditioned on ego-robot actions.

By Junwon Seo, Sushant Veer, Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Marco Pavone, Andrea Bajcsy

llms agents diffusion robotics multimodal benchmarks

arXiv AI

Jun 2

Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

arXiv:2605. 30280v2 Announce Type: replace-cross Abstract: Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments.

By Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, Xuhong Huang, Pei Lin, Junyang Lin, Dayiheng Liu, Shuai Bai, Jingren Zhou, Jiazhao Zhang, Haoqi Yuan, Gengze Zhou, Hang Yin, Ye Wang, Yiyang Huang, Zixing Lei, Wujian Peng, Delin Chen, Yingming Zheng, Jingyang Fan, Xianwei Zhuang, Xin Zhou, Haoyang Li, Anzhe Chen, Tong Zhang, Xuejing Liu, Yuchong Sun, Ruizhe Chen, Zhaohai Li, Chenxu L\"u, Zhibo Yang, Tao Yu, Xionghui Chen

robotics multimodal benchmarks

arXiv AI

Jun 2

Continuous Reasoning for Vision-Language-Action

arXiv:2606. 00229v1 Announce Type: cross Abstract: Natural language is a powerful reasoning medium for language and vision-language models, but it is mismatched to the granularity of continuous control.

By Yueh-Hua Wu, Tatsuya Matsushima, Kei Ota

llms robotics multimodal

arXiv AI

Jun 2

SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes

arXiv:2602. 09153v2 Announce Type: replace-cross Abstract: Simulation has become a key tool for training and evaluating home robots at scale, yet existing environments fail to capture the diversity and physical complexity of real indoor spaces.

By Nicholas Pfaff, Thomas Cohn, Sergey Zakharov, Rick Cory, Russ Tedrake

agents robotics multimodal

arXiv AI

Jun 2

Genie 4D: Semantic-Prior-Guided 4D Dynamic Scene Reconstruction

arXiv:2604. 09877v2 Announce Type: replace-cross Abstract: At the intersection of computer vision and robotic perception, 4D reconstruction of dynamic scenes connects low-level geometric sensing with high-level semantic understanding.

By Yiru Yang, Zhuojie Wu, Nishant Kumar Singh, Max Schulthess

rag diffusion computer-vision robotics benchmarks

arXiv AI

Jun 2

MindClaw: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention

arXiv:2606. 01063v1 Announce Type: new Abstract: Theory of Mind (ToM) enables an agent to reason about another actor's beliefs, goals, and intentions, which is essential for human-centered embodied assistance.

By Ruoxuan Zhang, Qiaoqiao Wan, Zhengguang Wang, Chenghao Yu, Hongxia Xie, Jianlong Fu, Wen-Huang Cheng

agents nlp robotics multimodal benchmarks

arXiv AI

Jun 2

ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priors

arXiv:2603. 15956v3 Announce Type: replace-cross Abstract: Learning generalizable and robust behavior cloning policies requires large volumes of high-quality robotics data.

By Zifan Xu, Ran Gong, Maria Vittoria Minniti, Kausik Sivakumar, Ahmet Salih Gundogdu, Eric Rosen, Riedana Yan, Tushar Kusnur, Zixing Wang, Di Deng, Peter Stone, Xiaohan Zhang, Karl Schmeckpeper

llms diffusion reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jun 2

Neural Low-Discrepancy Sequences

arXiv:2510. 03745v2 Announce Type: replace Abstract: Low-discrepancy points are designed to efficiently fill the space in a uniform manner.

By Michael Etienne Van Huffel, Nathan Kirk, Makram Chahine, Daniela Rus, T. Konstantin Rusch

computer-vision robotics fine-tuning

arXiv Machine Learning

Jun 2

Lagrangian Perturbation Diffusion Steering: Latent Reinforcement Learning for Generative Policies

arXiv:2606. 01151v1 Announce Type: new Abstract: Behavior cloning with high-capacity generative policies achieves strong imitation performance, but is often limited by demonstration coverage and distribution shift.

By Hikmet Simsir, Ozgur S. Oguz

diffusion reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 2

Interaction-Centered Intelligence: Toward Interaction as the Primary Unit of Analysis in Co-Creative AI and Human-AI Systems

arXiv:2606. 00807v1 Announce Type: new Abstract: Traditional artificial intelligence has largely conceptualized intelligence as isolated computation occurring within bounded agents.

By Nicholas Davis

agents robotics benchmarks

arXiv AI

Jun 2

From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data

arXiv:2606. 00054v1 Announce Type: cross Abstract: Recent progress in generalizable embodied control has been driven by large-scale pretraining of Vision-Language-Action (VLA) models.

By Zhiyuan Feng, Qixiu Li, Huizhi Liang, Rushuai Yang, Yichao Shen, Zhiying Du, Zhaowei Zhang, Yu Deng, Li Zhao, Hao Zhao, Zongqing Lu, Oier Mees, Marc Pollefeys, Jiaolong Yang, Baining Guo

robotics multimodal

arXiv AI

Jun 2

MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction

arXiv:2604. 10169v2 Announce Type: replace Abstract: Trajectory prediction is a key component of autonomous driving systems because future motions directly affect collision checking, behavior planning, and control.

By Wenchang Duan, Zhenguo Gao, Jinguo Xian, Yi Shi

agents robotics efficiency multimodal safety

arXiv Machine Learning

Jun 2

Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards

arXiv:2606. 02194v1 Announce Type: new Abstract: Distilling expert demonstration data into large generative models using behavioral cloning is a scalable approach to learning capable policies for robotic control, particularly for dexterous manipulation.

By Christian Scherer, Joe Watson, Theo Gruner, Daniel Palenicek, Ingmar Posner, Jan Peters

reinforcement-learning robotics fine-tuning

arXiv AI

Jun 2

Shape Your Body: Value Gradients for Multi-Embodiment Robot Design

arXiv:2606. 00702v1 Announce Type: cross Abstract: We propose to turn generalist multi-embodiment value functions into reusable models for robot design.

By Nico Bohlinger, Jan Peters

reinforcement-learning robotics

arXiv AI

Jun 2

"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills

arXiv:2602. 06547v3 Announce Type: replace-cross Abstract: LLM-based coding agents increasingly rely on third-party extensions called skills, which bundle natural language instructions and helper scripts that execute with full user privileges.

By Yi Liu, Zhihao Chen, Yanjun Zhang, Gelei Deng, Yuekang Li, Jianting Ning, Leo Yu Zhang

llms agents robotics safety

arXiv AI

Jun 2

SilentDrift: Exploiting Action Chunking for Stealthy Backdoor Attacks on Vision-Language-Action Models

arXiv:2601. 14323v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models are increasingly deployed in safety-critical robotic applications, yet their security vulnerabilities remain underexplored.

By Bingxin Xu, Yuzhang Shang, Binghui Wang, Emilio Ferrara

rag robotics multimodal safety

Get the daily briefing