Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models

The paper introduces ROBORMBENCH, a benchmark comprising 2,390 real‑robot trajectories, 21,673 verified paraphrases, and ground‑truth progress labels, to evaluate paraphrase robustness in vision‑language reward models (VLMs). It demonstrates that current VLMs often give different rewards for semantically equivalent goal descriptions, sometimes flipping a robot’s outcome from failure to success. The study finds that this instability is widespread, worsens with more divergent rewrites, and is not mitigated by model scale or explicit reasoning, though dedicated reward models trained with trajectory‑grounded supervision show greater stability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• reinforcement-learning
• robotics
• multimodal
• benchmarks