arXiv AI
Words to World: Benchmarking LLMs in Interactive Physical Environments
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios
arXiv:2511. 17649v4 Announce Type: replace-cross Abstract: Tangible control interfaces (TCIs), such as appliance panels, remotes, elevators, and embedded GUIs, are a fundamental component of everyday human-built environments.
arXiv Computation and Language
Sep 15
PhysMent: An Interactive Approach For LLM Reasoning In Physics Problems
arXiv:2609.13152v1 Announce Type: new Abstract: Large language models (LLMs) perform strongly on static science benchmarks, yet their ability to reason about the physical world through active experim...
arXiv AI
Oct 2
Probing an Embodied LLM: When Higher Observation Fidelity Hurts Problem Solving
The study investigates how different observation fidelities affect the problem‑solving performance of embodied large language model (LLM) agents. Using the Lockbox mechanical puzzle, agents were tested with raw RGB, RGB‑D, and perfect ground‑truth symbolic observations in both physical and simulated environments. Surprisingly, agents performed best with raw RGB input and worst with perfect observations; introducing moderate perceptual noise in simulation further improved success rates, suggesting that some errors can mitigate repetitive action loops.
arXiv AI
Oct 21
Plausible Future, Wrong Choice: Steering World-Action Models toward Intended Future
World-Action Models (WAMs) jointly predict future world states and robot actions for diverse manipulation tasks, but they can suffer from Stage Confusion—generating plausible yet misdirected rollouts that drift to the wrong task stage. TaskWAM introduces a closed-loop, stage-aware framework using Mode Tokens and State Tokens to represent fine-grained task requirements and execution progress, steering generation toward the intended future. The authors also present StageBench, a multi-stage manipulation benchmark, and show through extensive simulation and real-world experiments that TaskWAM reduces confusion, matches state-of-the-art performance, and transfers effectively across various robot embodiments.
arXiv AI
Oct 15
Embodied Zero-Shot Task Planning via Hardware-Aware LLM-VLM Integration for Grounded Robotic Manipulation
arXiv:2610.14106v1 Announce Type: cross Abstract: Large language models can translate natural-language instructions into plausible manipulation plans, but robotic execution requires physically ground...
Hugging Face Trending Papers
Oct 7
RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments
General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world. To investigate this, we...