Robotics and embodied AI

Manipulation, locomotion, sim-to-real transfer and autonomous driving: learning systems that have to survive physics.

3,943 stories · RSS feed

arXiv Computer Vision
Sep 10

VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models

arXiv:2609.09396v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...

By Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain, Lap Fung Chan, John Suchanek, Yu Wang, Varun Praveen, Tomasz Kornuta, Vidya Nariyambut Murali
arXiv Machine Learning
Sep 10

LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Prediction

arXiv:2608.25757v4 Announce Type: replace-cross Abstract: Large-scale vision--language--action (VLA) policies have advanced generalist robot control, yet most remain stimulus-to-action black boxes: a...

By Jin Lou, Zhiyuan Jing, Xupeng Wang, Andong Chen, Xingdong Zhu, Yuexuan Li, Yuan Xu, Zhijie Zhu, Yingwei Ji, Wenpeng Nie, Renxing Feng, Liangliang Chen, Ying Chu, Jingyi Li, Jinyan Liu, Zhiqi Song, Jingxuan Zhu, Jidong Zhang, Yufei Liu, Boyang Xing, Lei Jiang, Yan Cui, Hongming Li, Yuchen Zhu
arXiv Machine Learning
Sep 10

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

arXiv:2609.08084v1 Announce Type: cross Abstract: Monocular depth estimation is a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computati...

By Igor Pavlovic, Thiemo Wandel, Anton Obukhov, Luca Bartolomei, Andrey Davydov, Fabio Tosi, Matteo Poggi, Sabine S\"usstrunk, Dengxin Dai
arXiv AI
Sep 10

SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

SpatialBlock introduces a synthetic dataset of 15,000 block‑stacking problems designed to improve spatial intelligence in Large Vision‑Language Models (LVLMs). The dataset covers 3D‑to‑2D projection, viewpoint transformation, and structural combination, and uses controlled color modulation to encourage anchor‑based reasoning. Experiments show that LVLMs trained on SpatialBlock outperform baselines and generalize to real‑world spatial tasks, despite the dataset’s synthetic and compact nature.

By Soohyun Ryu, Sohee Kim, Eunho Yang
arXiv AI
Sep 10

PAN: A World Model for General, Actionable, and Long-Horizon World Simulation

arXiv:2511.09057v4 Announce Type: replace-cross Abstract: A world model is a cognitive simulator of the real-world environment allowing biological agents to reason about how the world evolves, whethe...

By PAN Team, Zihan Liu, Yi Gu, Mingkai Deng, Guangyi Liu, Zeyu Feng, Qiyue Gao, Yiyan Hu, Benhao Huang, Yichi Yang, Kun Zhou, Jiannan Xiang, Zhiting Hu, Zhengzhong Liu, Eric P. Xing
arXiv AI
Sep 10

FPicker: Topology-Guided Evolution for Filament Tracing in Low-SNR Microscopy

FPicker is a topology-guided framework for filament tracing in low‑signal Cryo‑EM images. It combines a center‑endpoint representation with an open‑curve evolution module to model non‑cyclic connectivity, overcoming limitations of pixel‑wise segmenters, box‑based detectors, sequential trackers, and traditional active contours. On simulated benchmarks, FPicker improves mean spatio‑angular precision by over 40% and reduces topological gap rates by more than 60% under extreme noise, and it achieves state‑of‑the‑art performance on real EMPIAR data after fine‑tuning.

By Tingyin Zhao, Mingtao Huang, Yuan Shen
arXiv Machine Learning
Sep 10

Learning Acrobatic Flight from Preferences

The paper introduces Reward Ensemble under Confidence (REC), a probabilistic reward learning framework for preference-based reinforcement learning that models per‑timestep reward uncertainty using an ensemble of distributional reward models. REC incorporates uncertainty into the preference loss and uses model disagreement to drive exploration, achieving 88.4% of shaped‑reward performance on acrobatic quadrotor control versus 55.2% with standard Preference PPO. The authors train policies in simulation and transfer them zero‑shot to real quadrotors, demonstrating complex acrobatic maneuvers learned solely from human preference feedback, and validate REC on a continuous‑control benchmark.

By Colin Merk, Ismail Geles, Jiaxu Xing, Angel Romero, Giorgia Ramponi, Davide Scaramuzza
arXiv AI
Sep 10

How to Backdoor Image Knowledge Distillation

The paper demonstrates that image knowledge distillation can be backdoored even when the teacher model is clean, by poisoning the distillation dataset with triggered and manipulated images that the teacher already classifies as a target label. The attack, effective at poisoning rates as low as 10%, uses targeted adversarial perturbations and GAN-based class transitions to embed a backdoor into the student model while preserving its performance on clean data. The study highlights that the security of knowledge distillation depends not only on the teacher but also on the integrity of the distillation data.

By Qian Ma, Chen Wu, Prasenjit Mitra, Sencun Zhu