Hugging Face Blog

Pollen-Vision: Unified interface for Zero-Shot vision models in robotics

arXiv Computer Vision
Oct 8

Agentic Scene Policies

arXiv:2509.19571v2 Announce Type: replace-cross Abstract: Designing or learning robot policies that generalize zero-shot across a range of language instructions and objects is a core problem in robot...

By Sacha Morin, Kumaraditya Gupta, Mahtab Sandhu, Charlie Gauthier, Francesco Argenziano, Kirsty Ellis, Liam Paull
arXiv Machine Learning
Sep 24

Generalizable Robotic Insertion with World Models

The paper introduces a framework that uses world models to enable robotic insertion across diverse parts. By combining proprioceptive data with wrist‑mounted camera visuals, a single model is trained on up to 90 tasks, achieving 56% zero‑shot success on unseen objects versus 7% for a model‑free baseline. The approach scales with more training objects and can be fine‑tuned for improved data efficiency and performance.

By Nicklas Hansen, Iretiayo Akinola, Yijie Guo, Jie Xu, Bingjie Tang, Hao Su, Xiaolong Wang, Abhishek Gupta, Dieter Fox, Yashraj Narang
arXiv AI
Aug 28

Active sensing to characterize the heterogeneity of plant stress

The paper introduces an autonomous robotic platform that performs targeted chlorophyll fluorescence measurements on plant leaves. It integrates 3D plant reconstruction, geometric analysis, and motion planning to identify suitable leaf surfaces and generate collision‑free trajectories for a robotic manipulator. This system enables automated, repeatable, and spatially resolved physiological measurements that extend beyond passive imaging.

By Ayman Laaroussi, Peter Hanappe, David Colliaux
arXiv AI
Jul 1

MVP-Nav: Multi-layer Value Map Planner Navigator

arXiv:2606. 31919v1 Announce Type: cross Abstract: Zero-shot Object Goal Navigation (ZSON) with RGB-only perception poses a fundamental challenge for embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment.

By Wenyuan Xie, Shaokai Wu, Yijin Zhou, Yanbiao Ji, Guodong Zhang, Bayram Bayramli, Qiuchang Li, Xunchu Zhou, Yue Ding, Hongtao Lu
arXiv Computer Vision
Aug 24

From Simulation to the Real-World: An In-Field 6D Pose Dataset and Baseline for Robotic Strawberry Harvesting

The paper presents the first real‑world 6D pose ground‑truth dataset for red‑stage strawberries, collected from 12,040 images at an actual farm using indirect camera pose recovery and 3D bounding‑box annotation. It also introduces a synthetic dataset rendered in NVIDIA Isaac Sim with scene‑level realism and domain randomization. Experiments show that models trained solely on synthetic data do not transfer well to in‑field images, but adding a small amount of real data significantly improves both translation and rotation accuracy across various backbone encoders.

By Woojung Son (Department of Agricultural and Biological Engineering, University of Florida), Won Suk Lee (Department of Agricultural and Biological Engineering, University of Florida), Zijing Huang (Department of Agricultural and Biological Engineering, University of Florida), Daeun Choi (Department of Agricultural and Biological Engineering, University of Florida), Catia Silva (Department of Electrical and Computer Engineering, University of Florida), Yu She (Edwardson School of Industrial Engineering, Purdue University), Yan Gu (School of Mechanical Engineering, Purdue University)
arXiv AI
Jun 19

Human Universal Grasping

arXiv:2606. 17054v1 Announce Type: cross Abstract: Humans can grasp objects effortlessly, whereas multi-fingered robots are far from this level of generality.

By Kevin Yuanbo Wu, Tianxing Zhou, Isaac Tu, Billy Yan, Irmak Guzey, David Fouhey, Dandan Shan, Lerrel Pinto
arXiv AI
Sep 4

AnyBox: Efficient Zero-Shot 9DoF Pose Estimation of Boxes for Robotic Manipulation

AnyBox is a zero‑shot framework that estimates the full 9DoF pose (6D pose plus 3D dimensions) of boxes from a single RGB‑D image, leveraging the geometric regularity of boxes. It alternates between pose and scale estimation, using a binary search guided by the discrepancy between a reprojected template and the observed mask, and employs a depth‑consistency filter and an early‑stopping rule to prune implausible hypotheses. On public benchmarks and a warehouse dataset, AnyBox improves detection AP by up to 36 points and boosts robotic box‑shelving success by 28%.

By Yintao Ma, Sajjad Pakdamansavoji, Charles Eret, Rui Heng Yang, Xuan Zhao, Yingxue Zhang, Tongtong Cao, Amir Rasouli