← Back to all news
arXiv AI By Alessandro Nazzari, Nathan Cerisara, Dorian Tonnis, Raina Zakir, Lorenzo Labarile, Weixu Zhu, Marco Dorigo, Mary Katherine Heinrich

Auto-HSI: Personalized human control of a robot swarm on demand by using LLMs for online automatic code generation

Read the original on arXiv AI →

Auto-HSI is a system that creates personalized human‑swarm interaction interfaces on demand using large language models to automatically generate code from natural language descriptions and gesture demonstrations. The prototype enables untrained operators to control a swarm of 50 simulated robots with one‑ or two‑hand gestures, allowing teleoperation of motion, formation shape, and shape deformation. Experiments demonstrate the system’s gesture tracking, code generation, and live operation capabilities, including real‑time updates and deployment on real robots.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 10

RoboCousin: Build Your Own Simulation Playground for Robust Bimanual Robotic Manipulation

RoboCousin is an extensible simulation platform that transforms user-provided observations into reusable assets, scenes, and expert trajectories for bimanual robotic manipulation. It converts object images into simulation-ready models with visual, collision, semantic, and physical metadata, automatically generates grasp candidates, and builds digital cousins that vary objects, backgrounds, layouts, and language instructions while preserving task-relevant affordances. The platform supports both tabletop and room-level scene construction, and the authors release RoboCousin-OBD with over 3,000 annotated objects and 50 backgrounds, generating more than one million expert trajectories across 50 tasks, with simulation and real-robot experiments demonstrating comparable annotation quality and effective sim-to-real transfer.

By Jingxuan Zhu, Jingyi Li, LiangLiang Chen, Zhiyuan Jing, Jidong Zhang, Hongming Li
robotics
More like this →