The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper investigates whether vision‑language‑action (VLA) models truly follow How‑level instructions that detail how to perform a task. The authors find that current VLAs do not benefit from such instructions and that task success actually decreases. They propose training with segment‑level enriched instructions and a new benchmark, SIMPLER‑IN, to better assess How‑level instruction following, but conclude that this remains an open challenge.
arXiv:2606. 12299v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone.
arXiv:2603. 06001v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models enable robots to perform manipulation tasks directly from natural language instructions and are increasingly viewed as a foundation for generalist robotic policies.
arXiv:2610.10526v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-langua...
DeicticVLA unifies three instruction modes—Language Instruction, Vision‑Language Instruction, and Visual Instruction—into a single text prompt and deictic mask framework, allowing a single pretrained Vision‑Language‑Action model to handle all modes. The approach uses text‑prompt completion and deictic gesture grounding, and evaluates various visual prompting methods and training strategies in simulation and real‑world tasks. Results show that two‑stage training improves deictic mask usage, and that Vision‑Language and Visual Instruction outperform Language Instruction on unseen expressions, appearance changes, and novel objects, achieving 100% success on unseen categories.
The paper introduces CRAFT, a method for improving compositional generalization in vision‑language‑action (VLA) models. It addresses the issue where models rely on visual shortcuts during fine‑tuning, leading them to execute demonstrated skill combinations that match observations rather than the instructed ones. By training with counterfactual instruction–observation pairs and transferring supervision through reusable skill representations, CRAFT enhances success on unseen skill combinations while preserving performance on demonstrated ones across multiple VLA models and benchmarks.