arXiv Computer Vision
STRIKE: Learning Visual State Transitions for Physical World Modeling
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Sep 17
StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
arXiv:2609.18430v1 Announce Type: new Abstract: Modeling physical dynamics, including how objects move, interact, and change state, is central to video world models for embodied AI. We present StrucP...
arXiv Computer Vision
6d ago
PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
PDRVLA is a Vision‑Language‑Action model that incorporates a lightweight Physical Dynamics Module to extract physical tokens from vision‑language features, enabling action generation without online video synthesis. The module predicts future RGB, depth, and optical‑flow features, guided by auxiliary supervision from a frozen pretrained video world model. Trained solely on publicly available data augmented with human‑to‑robot synthesis, subtask annotation, and embodied chain‑of‑thought (ECoT) annotation, PDRVLA achieves high success rates across four simulation benchmarks, with ablations demonstrating the benefits of world‑model supervision and various conditioning strategies.
arXiv Computer Vision
Oct 16
VLA-$\Delta$: Learning Physical State Transition for Vision-Language-Action Models
VLA‑Δ introduces a framework that explicitly learns physical state transitions for vision‑language‑action models, using changes between current and future states as learning targets. The authors build a large‑scale pretraining dataset with physical state‑transition annotations that cover task‑relevant semantics and visual motion changes, providing direct supervision for continuous action generation. The resulting model performs competitively on LIBERO and RoboTwin benchmarks and outperforms π₀.₅ in real‑robot experiments, while requiring no extra inference overhead.
arXiv AI
Aug 25
Inferring Action from Future Latent State for Robotic Manipulation
arXiv:2608.22067v1 Announce Type: cross Abstract: World-Action Models (WAMs) build robot control on video-generation backbones, which jointly predict dense future visual trajectories and robot action...
arXiv Computer Vision
Sep 15
PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models
PhysBrain 1.5 is a unified vision‑language model that learns to understand physical environments, generate actions, and predict future states by encoding language, end‑effector motion, and dense visual targets as discrete sequences and training them with autoregressive next‑token prediction. The model is pre‑trained on human interaction videos and fine‑tuned on human demonstrations, robot trajectories, and simulated experience, achieving an average score of 72.5 across 28 embodied understanding benchmarks and outperforming other open‑source models on 14 of them. It also demonstrates the ability to produce end‑effector trajectories and predict future scenes with spatially aligned RGB, depth, and robot‑mask outputs.
arXiv AI
Jul 31
CG-World: A Large-Scale World-State Dataset and Protocol for World Models
arXiv:2607. 26452v1 Announce Type: new Abstract: World models must learn the joint dynamics of states, actions, events, and observations, yet existing video, robotics, and simulation datasets usually capture only part of this structure.