Robotics and embodied AI

Manipulation, locomotion, sim-to-real transfer and autonomous driving: learning systems that have to survive physics.

2,040 stories · RSS feed

DeepMind Blog

Jul 13

Empowering India’s next generation of innovators with ATL Saathi

Google and AIM launched ATL Saathi, a Gemini-powered AI tool empowering Indian educators in robotics labs.

llms robotics

Hugging Face Trending Papers

Jul 13

A Glimpse into Long-term Physical Coexistence with Intelligent Robots

Long-term physical coexistence with intelligent robots requires more than capable robot policies. A persistent robotic assistant must support diverse user-facing interfaces, maintain long-horizon memory of people and preferences, coordinate across robot embodiments, and translate human intent into safe physical execution.

agents robotics

arXiv AI

Jul 13

Prompt-Driven Exploration

arXiv:2607. 08837v1 Announce Type: cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers.

By Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed, Ruiyang Luo, Nitish Dashora, Richard Li, Pulkit Agrawal, Zhang-Wei Hong

llms reinforcement-learning robotics multimodal

arXiv AI

Jul 13

Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning

arXiv:2607. 09336v1 Announce Type: cross Abstract: Diffusion-based trajectory planners have shown strong performance in offline reinforcement learning, but their iterative denoising process often incurs high inference cost.

By Guanquan Wang, Yoshimasa Tsuruoka

diffusion reinforcement-learning robotics efficiency benchmarks

arXiv AI

Jul 13

Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks

arXiv:2607. 09330v1 Announce Type: new Abstract: Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in physical artificial intelligence such as smart factories, warehouses, and service robotics.

By Nuocheng Yang, Sihua Wang, Zihan Chen, Tony Q. S. Quek, Changchuan Yin

llms agents reinforcement-learning robotics

arXiv AI

Jul 13

Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge

arXiv:2511. 20297v2 Announce Type: replace Abstract: Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from experience: each new session rediscovers solutions from scratch.

By Shashank Kirtania, Param Biyani, Priyanshu Gupta, Yasharth Bajpai, Roshni Iyer, Sumit Gulwani, Gustavo Soares

llms rag agents robotics benchmarks

arXiv AI

Jul 13

Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching

arXiv:2506. 22784v2 Announce Type: replace-cross Abstract: Point-pixel registration between LiDAR point clouds and camera images is a fundamental yet challenging task in autonomous driving and robotic perception.

By Yu Han, Zhiwei Huang, Yanting Zhang, Fangjun Ding, Shen Cai, Xiaoyu Tang, Yanchao Dong, Rui Fan

agents robotics efficiency multimodal benchmarks

arXiv AI

Jul 13

PAC-ACT: Post-training Actor-Critic for Action Chunking Transformers

arXiv:2607. 09590v1 Announce Type: cross Abstract: Precision industrial contact manipulation requires reliable robot policies under pose perturbations and contact-force constraints.

By Yujie Pang, Zudong Li

llms rag robotics fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 13

ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning

arXiv:2512. 16861v2 Announce Type: replace-cross Abstract: Long-horizon manipulation has been a long-standing challenge in the robotics community.

By Zihan Zhou, Animesh Garg, Ajay Mandlekar, Caelan Garrett

reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jul 13

Tactile and Vision Conditioned Contact-Centric Control for Whole-Arm Manipulation

arXiv:2607. 09218v1 Announce Type: cross Abstract: Whole-arm manipulation involves direct contact with the environment while the robot completes a task by distributing contact across multiple links as contacts form, slide, and break.

By Rishabh Madan, Angchen Xie, Samantha Saak, Andres Blanco, Dohyeok Lee, Sarah Grace Brown, Yunting Yan, Mark Zolotas, Jose Barreiros, Tapomayukh Bhattacharjee

robotics benchmarks

arXiv AI

Jul 13

Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference

arXiv:2607. 09520v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood.

By Junfei Zhan, Haoxun Shen, Mingang Guo, Zixuan Huang, Tengjiao He

llms robotics efficiency multimodal

arXiv AI

Jul 13

Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitors

arXiv:2505. 14300v2 Announce Type: replace Abstract: White-box monitoring is increasingly adopted as an auditing tool as Large Language Models (LLMs) are deployed in daily operations to ensure safe model behavior.

By Maheep Chaudhary, Fazl Barez

llms agents robotics benchmarks

arXiv AI

Jul 13

Knowledge-Based Design Requirements for Generative Social Robots in Higher Education

arXiv:2602. 12873v5 Announce Type: replace-cross Abstract: Generative social robots (GSRs) powered by large language models enable adaptive, conversational tutoring but also introduce risks such as misinformation, overreliance, and privacy violations.

By Stephan Vonschallen, Dominique Oberle, Theresa Schmiedel, Friederike Eyssel

llms agents robotics safety

arXiv Machine Learning

Jul 13

GenVid2Robot: From Video Generation to Robot Manipulation via Rigid-Geometric Consistency

arXiv:2607. 09191v1 Announce Type: cross Abstract: Generated videos provide useful visual motion priors for robot manipulation, but their visual plausibility does not imply physical executability.

By Haohui Huang, Xi Yuan, Panpan Liao, Tao Teng, Chenguang Yang, Jing Guo, Yi Guo

diffusion robotics

arXiv Machine Learning

Jul 13

SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions

arXiv:2607. 08925v1 Announce Type: new Abstract: Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do.

By Elham Daneshmand, Majid Khadiv, Glen Berseth, Hsiu-Chin Lin

agents reinforcement-learning robotics benchmarks safety

arXiv AI

Jul 13

CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding

arXiv:2607. 08974v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control.

By Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya, Kuniaki Saito, Cristian C. Beltran-Hernandez, Mai Nishimura

robotics fine-tuning multimodal

arXiv AI

Jul 13

Video Generation Models are General-Purpose Vision Learners

arXiv:2607. 09024v1 Announce Type: cross Abstract: Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models.

By Letian Wang, Chuhan Zhang, Rishabh Kabra, Jasper Uijlings, Steven Waslander, Andrew Zisserman, Joao Carreira, Kaiming He, Misha Andriluka, Eduard Gabriel Bazavan, Andrei Zanfir, Cristian Sminchisescu

diffusion computer-vision nlp robotics multimodal benchmarks safety

arXiv AI

Jul 13

What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility

arXiv:2607. 09503v1 Announce Type: cross Abstract: A fundamental challenge in 3D reconstruction and robotic localization is co-visibility: determining which image pairs share overlapping visible surfaces, particularly in scenarios with minimal overlap.

By Filippo Ziliotto, Luciano Serafini, Lamberto Ballan, Tommaso Campari

llms robotics benchmarks

arXiv Machine Learning

Jul 13

Multimodal Scenario Similarity Search for Autonomous Driving

arXiv:2607. 09428v1 Announce Type: cross Abstract: Large-scale autonomous-driving datasets contain vast numbers of recorded scenarios, creating a need for efficient retrieval methods that can identify situations similar to a given query.

By Tam\'as Matuszka, Andr\'as Tam\'asy, Bal\'azs Szol\'ar

llms rag agents robotics multimodal

arXiv AI

Jul 13

Embodied Multi-Agent Coordination by Aligning World Models Through Dialogue

arXiv:2605. 12920v3 Announce Type: replace-cross Abstract: Effective collaboration between embodied agents requires more than acting in a shared environment; it demands communication grounded in each agent's evolving understanding of the world.

By Vardhan Dongre, Dilek Hakkani-T\"ur

llms agents robotics benchmarks safety