← Back to all news
Hugging Face Trending Papers

Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use

The paper investigates how vision–language models (VLMs) diagnose failures in bimanual tool‑use tasks, where failures can arise from tool slip, timing mismatches between hands, or combined causes. It evaluates which sensing channels enable a diagnoser to identify each failure type, finding that tool slip is detectable on the tool hand, timing delays are observable on both arms, and some combined failures remain indistinguishable from passive sensing. The study also shows that a repair policy that escalates when the cause is unclear can match the performance of a policy with full cause knowledge, and that a VLM’s internal state captures the late hand almost perfectly, though the answer depends on question wording.

Related stories

arXiv AI
Oct 28

Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use

The paper investigates how different sensing channels help diagnose failures in a simulated bimanual tool‑use task. It finds that a tool’s slip is best detected by the tool hand, while the identity of the late hand can be inferred from both arms but not reliably from a single hand’s sensor. A repair policy that escalates when the cause is unclear can fix as many failures as one that knows the exact cause within two attempts.

By Baptiste Bonin, Caro Strickland, Audrey Durand
llmsagentsroboticsmultimodal
More like this →
arXiv AI
Aug 26

Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail

The study investigates why small language model agents tend to repeat a tool call that just failed. By recording the failed call and its error message in the transcript, the authors measure a negative corrective gain—agents are more likely to repeat the failed action, with a drop of about 1.03 nats per token. The problem is traced to the harness design rather than the model’s understanding of errors, and the authors show that replacing the verbatim call with a runtime-generated description of the failure can reduce this backfiring effect by 76%.

By Esmail Gumaan
llmsagents
More like this →
arXiv Computation and Language
Aug 31

Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models

The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.

By Justin Bronder
llms
More like this →
arXiv AI
Oct 27

What Can One Sentence Fix? A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

The paper investigates whether a single descriptive sentence can correct failures in frozen vision‑language‑action (VLA) policies. By labeling failures in the LIBERO‑90 benchmark, the authors identify four failure modes, finding that only execution misalignment—where the policy reaches the correct target but misjudges distance or force—can be remedied with a concise human‑generated sentence. This sentence, parsed by an LLM into a bias on the policy’s actions, successfully restores performance on both simulated and real‑world tasks, while automated language models fail to achieve similar gains.

By Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh
llmsroboticsfine-tuningmultimodalsafety
More like this →