Puffin-World is a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation without external offline modules. It jointly models physics, geometry, and appearance as native world states and uses a unified Omni-Camera representation to support diverse tasks and flexible motions. The framework also propagates physical dynamics across future frames, couples appearance and geometry in a single generative process, and scales to complex scenarios with the Puffin-16M dataset of 15 million vision‑language‑camera triplets and 1 million trajectories.
By Kang Liao, Yihang Luo, Xiao-Ming Wu, Linyi Jin, Size Wu, Chunyu Lin, Yao Zhao, Fei Wang, Wei Li, Chen Change Loy
arXiv:2610.28631v1 Announce Type: new
Abstract: Current world models for explicit world simulation typically follow two distinct pathways: (1) video generative models (e.g., Sora), which simulate rea...
By Weiqi Zhang, Wenyuan Zhang, Yu-Shen Liu, Junsheng Zhou
arXiv:2603. 03482v2 Announce Type: replace-cross Abstract: Interactive world models continually generate video by responding to a user's actions, enabling open-ended generation capabilities.
By Samuel Garcin, Thomas Walker, Steven McDonagh, Tim Pearce, Hakan Bilen, Tianyu He, Kaixin Wang, Jiang Bian
Predicting how the world evolves in response to robot actions is central to policy learning, evaluation, and planning. However, many action-conditioned world models predict future observations in 2D i...
EvoWorld is a world model that generates panoramic video from a single image by combining a video generator with fine‑grained view control, a feedforward transformer that evolves an explicit 3D reconstruction, and a synthesis step that conditions on geometric reprojections from this evolving 3D memory. The model uses the evolving 3D reconstruction as spatial guidance, projecting geometry onto target viewpoints to provide rich spatial cues that improve visual realism and geometric consistency. A new benchmark covering synthetic outdoor, Habitat indoor, and real‑world scenarios evaluates long‑range exploration, loop‑closure detection, and spatial coherence, showing that EvoWorld’s evolving 3D memory enhances visual fidelity and scene coherence over existing methods.
By Jiahao Wang, Luoxin Ye, TaiMing Lu, Junfei Xiao, Jiahan Zhang, Yuxiang Guo, Xijun Liu, Rama Chellappa, Cheng Peng, Alan Yuille, Jieneng Chen
arXiv:2606.18250v2 Announce Type: replace
Abstract: Forecasting the evolution of dynamic environments is crucial for autonomous agents. While generative world models have achieved high photorealism i...
By Nils Morbitzer, Jonathan Evers, Artem Savkin, Thomas Stauner, Nassir Navab, Federico Tombari, Stefano Gasperini