← Back to all news
arXiv AI By Yiyang Du, Zhanqiu Guo, Haochen Zhang, Annabella Macaluso, Xin Ye, Liu Ren, Yonatan Bisk, Chenyan Xiong

EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training

Read the original on arXiv AI →

EmbodiedMidtrain bridges the gap between Vision‑Language Models (VLMs) and Vision‑Language‑Action Models (VLAs) by first analyzing the distribution mismatch between their data. It introduces a lightweight proximity estimator that selects VLA‑aligned samples from a large VLM pool, creating the EmbodiedMidtrain‑1M dataset used to mid‑train VLMs before VLA fine‑tuning. Experiments on simulated and real‑world robot tasks show consistent performance gains across different VLM backbones and VLA action heads, with early training steps already benefiting from the improved initialization.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Aug 31

Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models

The paper introduces VLAct, a Vision‑Language‑Action model that focuses on representation‑centric continued pre‑training rather than merely scaling robot data. VLAct is trained on diverse, multi‑embodiment robot data and preserves a broad VLM prior while encouraging shared action semantics across embodiments. Experiments across simulation, real‑world, and unseen‑embodiment settings show that VLAct consistently outperforms existing industrial VLA systems, achieving high success rates with only a modest compute budget and open‑source data.

By Senqiao Yang, Chengyao Wang, Yuxin Chen, Zixuan Wang, Longxiang Tang, Haokun Gui, Jinhui Ye, Changsheng Lu, Xiaoyang Wu, Mingkang Zhu, Pengguang Chen, Shu Liu, Zhuotao Tian, Hengshuang Zhao, Bei Yu, Jiaya Jia
roboticsfine-tuningmultimodal
More like this →
arXiv AI
Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen
llmscomputer-visionroboticsfine-tuningmultimodalbenchmarks
More like this →
arXiv AI
Jul 3

VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment

arXiv:2607. 01586v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) have recently advanced robotic manipulation, yet the effects of different robot-data pre-training paradigms remain difficult to compare because existing models often differ in architecture, data, action space, and evaluation protocol.

By Guoyang Xia, Fengfa Li, Hongjin Ji, Lei Ren, Fangxiang Feng, Kun Zhan, Yan Xie
roboticsmultimodalbenchmarkssafety
More like this →