Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use

The paper investigates how vision–language models (VLMs) diagnose failures in bimanual tool‑use tasks, where failures can arise from tool slip, timing mismatches between hands, or combined causes. It evaluates which sensing channels enable a diagnoser to identify each failure type, finding that tool slip is detectable on the tool hand, timing delays are observable on both arms, and some combined failures remain indistinguishable from passive sensing. The study also shows that a repair policy that escalates when the cause is unclear can match the performance of a policy with full cause knowledge, and that a VLM’s internal state captures the late hand almost perfectly, though the answer depends on question wording.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

• llms
• agents
• robotics
• multimodal