Robotics and embodied AI

Manipulation, locomotion, sim-to-real transfer and autonomous driving: learning systems that have to survive physics.

4,111 stories · RSS feed

arXiv Computer Vision
Aug 26

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

GlanceWAM introduces a sparse test‑time imagination approach for world‑action models that decouples visual imagination from control. By asynchronously generating a single lookahead frame on a slow clock and decoding action chunks at a 48 ms control rate purely in latent space, it avoids latency while maintaining high success. The method achieves 72.2 % on the RoboCasa kitchen benchmark and 99.0 % on LIBERO, running 24× faster than synchronous baselines.

By Linhan Wang, Zijian An, Mingyuan Zhang, Chen Dai, Yi Xu, Can Cui, Zichong Yang, Yinlin Chen, Lifeng Zhou, Chang-Tien Lu
arXiv AI
Aug 26

Resilience Matters for Embodied Agents System: New Metrics, Systematic Evaluation, and Optimization

The paper introduces a new resilience evaluation framework for Embodied Agents Systems (EAS), defining metrics for Rebound, Stability, and Graceful Extensibility. It demonstrates that traditional outcome-centric metrics obscure dynamic behaviors, while the proposed metrics reveal process-level distinctions such as recovery costs and instability across 400 household tasks with 10 EAS. Optimizing with these metrics reduces recovery costs and improves stability and extensibility, highlighting a trade‑off among resilience characteristics that should guide deployment-specific system design.

By Yapeng Liu, Yuanzhao Zhai, Xudong Gong, Dawei Feng, Bo Ding, Lin Wang, Huaimin Wang
arXiv AI
Aug 26

Quantifying System-Level Harms from AI Adoption in Complex Sociotechnical Systems

The paper proposes a framework that connects structured hazard analysis, component-level testing, and probabilistic system modelling to assess system-level harms from AI in complex sociotechnical systems. It demonstrates the approach using the UK's Real Time Gross Settlement system, showing how adversarial inputs to LLM-based trading can shift AI behaviour, reduce system resilience, and increase the likelihood of cascading bank failures. The framework aims to provide a traceable pathway from model behaviour to systemic outcomes, enabling evidence-based governance of AI in critical infrastructure.

By Paul Vautravers, Oliver Chalkley, Gabriel Downer, Kate S, Damian Ruck
arXiv Computer Vision
Aug 26

O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics

O3N is a novel framework that performs open‑vocabulary occupancy prediction from a single omnidirectional RGB image. It introduces a polar‑spiral voxel embedding (PsM) for continuous 360° spatial representation, an Occupancy Cost Aggregation (OCA) module that unifies geometric and semantic supervision, and a Natural Modality Alignment (NMA) pathway that aligns visual, voxel, and text features. Experiments show state‑of‑the‑art results on QuadOcc and Human360Occ benchmarks, with strong cross‑scene generalization and semantic scalability.

By Mengfei Duan, Hao Shi, Fei Teng, Guoqiang Zhao, Yuheng Zhang, Zhiyong Li, Kailun Yang
arXiv AI
Aug 25

Continual Uncertainty Learning for Robust Control of Nonlinear Systems with Multiple Heterogeneous Uncertainties

The paper introduces Continual Uncertainty Learning (CUL), a curriculum-based continual learning framework that decomposes robust control of nonlinear systems with multiple heterogeneous uncertainties into a sequence of tasks. By progressively expanding and diversifying plant uncertainties and applying memory-efficient anti-forgetting regularization, CUL enables a policy to acquire strategies for each uncertainty sequentially while a model-based controller provides a shared baseline performance. Applied to an active vibration controller for automotive powertrains, the approach demonstrates robustness to structural nonlinearities and dynamic variations, improving control performance and sample efficiency.

By Heisei Yonezawa, Ansei Yonezawa, Itsuro Kajiwara
arXiv Computer Vision
Aug 25

GuideFetch: A Task Coordination Framework for Concurrent Navigation and Object Retrieval in Assistive Robot Dogs

arXiv:2608.18292v2 Announce Type: replace-cross Abstract: Consider one robot guide dog escorting a blind user to a seat while a second retrieves and delivers an object. We introduce \textsc{GuideFetc...

By Qian Yin, Ruiping Liu, Kunyu Peng, Jianxiang Man, Isik Baran Sandan, Junwei Zheng, Yufan Chen, Di Wen, Kailun Yang, Rainer Stiefelhagen