Robotics and embodied AI

Manipulation, locomotion, sim-to-real transfer and autonomous driving: learning systems that have to survive physics.

4,111 stories · RSS feed

arXiv AI
Aug 26

Pivot-and-Station Multi-Agent Path Finding: Solvability, Complexity, and Algorithms

The paper introduces Pivot-and-Station Multi-Agent Path Finding (PS‑MAPF), a variant of MAPF where a subset of agents must visit interchangeable pivots before all agents occupy anonymous stations. It provides a full solvability characterization: every instance on a 2‑edge‑connected graph is solvable, and for arbitrary connected graphs a structural effective‑distance measure relative to unoccupied vertices gives a necessary and sufficient condition. The authors prove that minimizing station‑makespan or station‑flowtime is NP‑hard even with a single pivot, and present three algorithms—a complete baseline, a SAT‑based optimal solver, and Pivot‑Prioritized Planning (PPP), which solves 74‑89% of benchmark instances with significantly lower makespan and flowtime than the baseline.

By Andrea Di Nezza, Mihir Patel, Fabio Fagnani, Sara Bernardini
arXiv Computer Vision
Aug 26

Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning

arXiv:2608.24885v1 Announce Type: cross Abstract: Action-conditioned world models are increasingly used as learned simulators for policy evaluation and improvement, yet their effectiveness rests on a...

By Sixiang Chen, Jiaming Liu, Jixian Wu, Yichen Guo, Tinghao Wang, Siyuan Qian, Hao Chen, Jiajun Cao, Jian Tang, Shanghang Zhang
arXiv Machine Learning
Aug 26

A Bayesian Learning Approach for Drone Coverage Network: A Case Study on Cardiac Arrest in Scotland

arXiv:2603.23134v2 Announce Type: replace Abstract: Drones are becoming popular as a complementary system for Emergency Medical Services (EMS). Although several pilot studies and flight trials have s...

By Tathagata Basu, Edoardo Patelli, Gianluca Filippi, Ben Parsonage, Christy Maddock, Massimiliano Vasile, Marco Fossati, Adam Loyd, Shaun Marshall, Paul Gowens
arXiv Machine Learning
Aug 26

WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation

arXiv:2608.24479v1 Announce Type: new Abstract: Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for...

By Zihao Wu, Hongyao Tang, Yi Ma, Huizhong Song, Pengyi Li, Yifu Yuan, Fei Ni, Jinyi Liu, Wei Wei, Jianrong Wang, Yan Zheng, Jianye Hao
arXiv AI
Aug 26

Hierarchical Skill Retrieval for Data-Efficient Adaptation of Vision-Language-Action Models

The paper introduces Hierarchical Skill Retrieval (HSR), a framework that decomposes a target manipulation task into candidate skill sequences and evaluates each plan for semantic plausibility and skill reliability. HSR combines subtask-level language retrieval with behavior-feature reranking to select demonstrations that are both relevant and compatible with the target task, followed by a two-stage pretraining and finetuning pipeline for policy adaptation. Experiments on the LIBERO benchmark and real-world robot tasks show that HSR improves average success rates by 10.3% and 21.3% over the strongest baseline, demonstrating the effectiveness of structured skill-level retrieval for data-efficient Vision‑Language‑Action adaptation.

By Haoran Hao, Shahram Najam Syed, Jeff Schneider, Jeffrey Ichnowski
arXiv AI
Aug 26

A Human-Factors Guided Cognitive Model of Visuospatial Complexity in Embodied Active Vision

The paper introduces a new framework for analyzing multimodal data—visual, auditory, and spatial—by focusing on the complexity of embodied perception and interaction in dynamic, naturalistic settings. It expands upon prior visual complexity work to categorize diverse attributes (quantitative, structural, dynamic, auditory, interactional) that together describe multimodal complexity. The authors illustrate how this model can characterize visuospatial complexity in everyday driving and suggest its use for creating and evaluating benchmark datasets that prioritize cognitive human factors.

By Vasiliki Kondyli, Jakob Suchan, Mehul Bhatt
arXiv AI
Aug 26

STAIN-FL: Stealthy Targeted Attack Injection with Contextual Triggers in Federated Learning

The paper introduces STAIN-FL, a stealthy backdoor attack framework for federated video anomaly detection that uses natural surveillance conditions—such as low light, indoor settings, and crowd density—as contextual triggers. STAIN-FL manipulates anomaly labels and masks gradients to keep clean accuracy low while inducing trigger‑conditioned misclassification. Experiments on UCF‑Crime with I3D features show that sparse attacks remain undetectable, drop clean accuracy by less than 2%, yet achieve over 50% backdoor accuracy for hundreds of rounds under FedAvg and FedProx.

By Ashlinder Kaur, Purnima Murali Mohan, Zengxiang Li, Tram Truong-Huu
arXiv AI
Aug 26

Resilience Matters for Embodied Agents System: New Metrics, Systematic Evaluation, and Optimization

The paper introduces a new resilience evaluation framework for Embodied Agents Systems (EAS), defining metrics for Rebound, Stability, and Graceful Extensibility. It demonstrates that traditional outcome-centric metrics obscure dynamic behaviors, while the proposed metrics reveal process-level distinctions such as recovery costs and instability across 400 household tasks with 10 EAS. Optimizing with these metrics reduces recovery costs and improves stability and extensibility, highlighting a trade‑off among resilience characteristics that should guide deployment-specific system design.

By Yapeng Liu, Yuanzhao Zhai, Xudong Gong, Dawei Feng, Bo Ding, Lin Wang, Huaimin Wang
arXiv AI
Aug 26

Macro-Operator Generation and Predicate Selection for TAMP Operator Learning

The paper introduces a system that automatically generates macro-operators—composite actions that compress recurring sequences of individual actions—by discovering causally linked action pairs in training data. It also prunes unused predicates from the symbolic state, reducing the number of predicates evaluated at each search node. These combined optimizations shorten the effective planning horizon and yield up to a 4.6× speedup, enabling the solution of long sequential tasks that baseline methods cannot solve.

By Can Emir Bora, Emre Ugur
arXiv AI
Aug 26

PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Control

PonderPounce introduces a two‑stage system that leverages a multimodal large language model (MLLM) as an episode‑context engine for robot control. The System2 component, Ponder, accumulates observations, demonstrations, and prior cognition in its native causal context, producing subgoal text and reasoning. The System1 component, Pounce, uses the current observation, instruction, and proprioception, receiving only the newest cognition token and its age from Ponder; the pair is jointly trained end‑to‑end without a separate memory module, achieving real‑time action playback and outperforming baseline methods on RoboMME and RoboCasa‑DC benchmarks.

By Suhwan Choi, Jaeyoon Jung, Sungkyung Kim, Yunsung Lee, Youngjae Yu
arXiv Machine Learning
Aug 26

E2HiL: Entropy-Guided Sample Selection for Efficient Real-World Human-in-the-Loop Reinforcement Learning

arXiv:2601.19969v2 Announce Type: replace-cross Abstract: Human-in-the-loop guidance has emerged as an effective approach for accelerating online reinforcement learning (RL) in real-world manipulatio...

By Haoyuan Deng, Yudong Lin, Yuanjiang Xue, Haoyang Du, Qianzhun Wang, Boyang Zhou, Zhenyu Wu, Ziwei Wang
arXiv AI
Aug 26

Quantifying System-Level Harms from AI Adoption in Complex Sociotechnical Systems

The paper proposes a framework that connects structured hazard analysis, component-level testing, and probabilistic system modelling to assess system-level harms from AI in complex sociotechnical systems. It demonstrates the approach using the UK's Real Time Gross Settlement system, showing how adversarial inputs to LLM-based trading can shift AI behaviour, reduce system resilience, and increase the likelihood of cascading bank failures. The framework aims to provide a traceable pathway from model behaviour to systemic outcomes, enabling evidence-based governance of AI in critical infrastructure.

By Paul Vautravers, Oliver Chalkley, Gabriel Downer, Kate S, Damian Ruck
arXiv Computer Vision
Aug 26

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

GlanceWAM introduces a sparse test‑time imagination approach for world‑action models that decouples visual imagination from control. By asynchronously generating a single lookahead frame on a slow clock and decoding action chunks at a 48 ms control rate purely in latent space, it avoids latency while maintaining high success. The method achieves 72.2 % on the RoboCasa kitchen benchmark and 99.0 % on LIBERO, running 24× faster than synchronous baselines.

By Linhan Wang, Zijian An, Mingyuan Zhang, Chen Dai, Yi Xu, Can Cui, Zichong Yang, Yinlin Chen, Lifeng Zhou, Chang-Tien Lu
arXiv Computer Vision
Aug 26

ConsensusTAS: Self-Supervised Temporal Action Segmentation for Long-Horizon Construction Videos

ConsensusTAS is a self‑supervised, label‑free method for temporal action segmentation in long construction videos. It segments continuous video streams into distinct activity phases by leveraging internal consensus among candidate segmentations, and it has been evaluated on three public datasets, outperforming state‑of‑the‑art methods. In real‑world construction footage, the model successfully identified fine‑grained actions within bricklaying, and it can run on a CPU, making it suitable for mobile robotic platforms.

By Xiaoshan Zhou, Yafei Sun