arXiv Machine Learning

Active Real-World Factor-Based Evaluation for Generalist Robot Policies

arXiv:2607. 14439v1 Announce Type: new Abstract: Generalist robot manipulation policies trained on large, diverse datasets have shown remarkable promise across a wide range of tasks.

arXiv AI

1d ago

RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies

arXiv:2604. 09860v4 Announce Type: replace-cross Abstract: The pursuit of general-purpose robotics has yielded impressive foundation models, yet simulation-based benchmarking remains a bottleneck due to rapid performance saturation and a lack of true generalization testing.

By Jenai Xuning Yang, Rishit Dagli, Alex Zook, Hugo Hadfield, Ankit Goyal, Stan Birchfield, Fabio Ramos, Jonathan Tremblay

llms robotics benchmarks

arXiv AI

Jun 9

Benchmarking Vision-Language-Action Models on SO-101: Failure and Recovery Analysis

arXiv:2606. 08881v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have demonstrated strong generalization in robotic manipulation, yet existing evaluations are primarily conducted in simulation or on expensive robotic platforms, leaving their robustness on affordable real-world robots largely unexplored.

By Yi Yu, Xinchuan Qiu

robotics fine-tuning multimodal benchmarks

arXiv AI

Jul 7

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies

arXiv:2607. 04434v1 Announce Type: cross Abstract: Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities.

By Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Weijie Wan, Baijun Chen, Haoran Lu, Haowen Yan, Honghao Su, Zhiyang Dou, Kaixuan Wang, Dandan Zhang, Yunze Liu, Yan Qin, Qiwei Liang, Qiwei Wu, Zijian Lin, Wenwei Lin, Yuran Wang, Minghua He, Tianshu Wu, Ruihai Wu, Jingquan Zhou, Kai-Chong Lei, Haibao Yu, Yuanfeng Ji, Weiyang Jin, Guanyu Lin, Xiaofan Li, Qi Xiong, Renjing Xu, Zhongyu Li, Wenhao Chai, Enze Xie, Ziwei Wang, Yao Mu, Hao Dong, Wojciech Matusik, Mingyu Ding, Wenbo Ding, Ping Luo, Masayoshi Tomizuka

robotics benchmarks

arXiv AI

Jun 30

Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Models

arXiv:2603. 22876v2 Announce Type: replace-cross Abstract: Learning a generalist control policy for robotic manipulation typically relies on large-scale datasets.

By Ruixing Jin, Zicheng Zhu, Ruixiang Ouyang, Sheng Xu, Bo Yue, Zhizheng Wu, Guiliang Liu

reinforcement-learning robotics multimodal benchmarks

arXiv AI

Aug 6

RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulation

arXiv:2510. 17640v4 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have shown strong manipulation capability when trained with large-scale imitation learning datasets.

By Yuquan Xue, Guanxing Lu, Zhenyu Wu, Chuanrui Zhang, Bofang Jia, Zhengyi Gu, Ziwei Wang

robotics multimodal benchmarks

arXiv AI

Jun 17

Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement

arXiv:2606. 18247v1 Announce Type: cross Abstract: Robots deployed in the real world should learn from their experience and improve over time.

By Mingtong Zhang, Dhruv Shah

agents robotics fine-tuning

arXiv AI

Jun 10

A Practical Recipe Towards Improving Sim-and-Real Correlation for VLA Evaluation

arXiv:2606. 10366v1 Announce Type: cross Abstract: Simulation has become an essential tool for evaluating and improving vision-language-action (VLA) policies, offering scalable, reproducible, and controllable alternatives to costly real-world robot evaluation.

By Shuo Wang, Hanyuan Xu, Yingdong Hu, Fanqi Lin, Yang Gao

robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jul 7

Verifier-free Test-Time Sampling for Vision-Language-Action Models

arXiv:2510. 05681v2 Announce Type: replace-cross Abstract: Vision-Language-Action models (VLAs) have demonstrated remarkable performance in robot control.

By Suhyeok Jang, Dongyoung Kim, Changyeon Kim, Youngsuk Kim, Jinwoo Shin

robotics multimodal benchmarks

arXiv AI

Jul 8

From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation

arXiv:2603. 15600v2 Announce Type: replace-cross Abstract: Accurate process supervision remains a critical challenge for long-horizon robotic manipulation.

By Yibin Liu, Yaxing Lyu, Daqi Gao, Zhixuan Liang, Weiliang Tang, Shilong Mu, Xiaokang Yang, Yao Mu

reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jun 30

RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis

arXiv:2606. 28385v1 Announce Type: cross Abstract: Recent advances in robot world models enable synthetic video generation for embodied prediction and planning.

By Minh-Loi Nguyen, Nghiem Tuong Diep, Hung Khang Nguyen, Minh Le, Doanh Le Thien, Hoang H. Tran, Dung D. Le, Vu N. Duong, Daniel Sonntag, An Thai Le, Duy Minh Ho Nguyen, Vien Anh Ngo, Tran Van Nhiem

llms agents diffusion robotics multimodal benchmarks safety

arXiv AI

Jun 30

Critical Interval MSE: Toward Reliable Offline Validation for Robot Manipulation Policies

arXiv:2606. 29898v1 Announce Type: cross Abstract: Real-world evaluation is the gold standard for robot policies because it tests them against the physical conditions and deployment challenges they are ultimately designed to handle.

By Haoxu Huang, Tongsam Zheng, Yifan Chen, Jiacheng You, Yang Gao

robotics safety

arXiv AI

Jul 2

FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement

arXiv:2607. 01111v1 Announce Type: cross Abstract: Robot policies inevitably encounter failures when deployed in real environments.

By Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski, Jeff Schneider

diffusion robotics

Related stories