arXiv Computer Vision

AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

arXiv Computer Vision
Sep 30

MUGEN: Interactive Panoramic World Exploration via Camera Control

MUGEN is a large‑scale real‑world dataset of over 1,300 hours of 4K panoramic videos with rich semantic and geometric annotations, designed to support interactive 360° world exploration. Wan360 is a camera‑controllable panoramic video generation model built on MUGEN, featuring ERP‑aware components (periodic longitude RoPE, ERP‑aware padding, random roll yaw) and a panoramic Plücker embedding for camera motion. Together, they address gaps in data and model support for immersive, temporally coherent 360° video generation along user‑specified camera trajectories.

By Jiaming Tan, Zhen Li, Shuwei Shi, Minggui Teng, Siqi Yang, Yuwei Wu, Bo Zheng, Chuanhao Li, Kaipeng Zhang
arXiv AI
Jul 22

AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report

arXiv:2607. 18367v1 Announce Type: new Abstract: Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interactive environments from user inputs instantly.

By AlayaWorld Team, Kaipeng Zhang, Chuanhao Li, Yifan Zhan, Yongtao Ge, Yuanyang Yin, Jiaming Tan, Kang He, Liaoyuan Fan, Mingliang Zhai, Ruicong Liu, Xiaojie Xu, Xuangeng Chu, Zhen Li, Zhengyuan Lin, Zhixiang Wang, Zian Meng, Zihui Gao
Hugging Face Trending Papers
Jun 11

MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold

We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations.

arXiv Computer Vision
Oct 2

World Observer: Joint Actor-Observer Generation for Persistent World Modeling

World Observer introduces a joint actor‑observer framework for persistent world modeling, decoupling the agent’s view from panoramic observers that track regions outside the actor’s field of view. By warping from a shared panoramic source and using an Observer Sink for high‑resolution references, the method maintains accurate object states and dynamics even when they leave and re‑enter the actor’s view. The approach is evaluated with new world‑space metrics and a benchmark across real and synthetic scenes, showing improved out‑of‑view dynamics while preserving visual fidelity, camera control, and 3D adherence.

By Hyunwook Choi, Dahyun Chung, Hyunsung Kim, Siyoon Jin, Jinhyeok Choi, Junyoung Seo, Seungryong Kim
arXiv AI
6d ago

OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images

OmniShowcase introduces a method for generating omnidirectional videos of a specific subject from a handful of multi‑view images. It combines an automated data‑synthesis pipeline that creates training videos with diverse camera trajectories driven by the subject’s geometry, and a geometry‑conditioning technique that supplies the model with surface‑normal maps reconstructed from the reference views. The authors release Showcase‑4K, a large corpus of 3D‑asset‑based videos with 12 camera motions and 88 trajectory configurations, and ShowcaseBench, a held‑out evaluation set with source meshes and camera parameters.

By Junyeong Ahn, Geonwoo Kim, Kinam Kim, Gihwi Kim, Hoonjin Jung, Hyojin Jang, Jaegul Choo
arXiv Computer Vision
Oct 8

SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation

SPW-Nav is a streaming panoramic world model that generates one minute of 2K 360‑degree video in real time from a single panorama, guided by language instructions. It interprets each instruction as camera motion, applying spherical rotation decoupling, pose‑aligned conditioning, and a multi‑term memory to maintain continuity as instructions change. The authors also release SPW-NavSet, a dataset of panoramic videos with camera trajectories and verified instructions, and demonstrate that SPW-Nav surpasses prior panoramic generators in camera‑following accuracy and video quality while enabling on‑the‑fly instruction switching.

By Yunheng Liu, Ziqi Cai, Siqi Yang, Yimu Wang, Minggui Teng, Jiaming Tan, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi
arXiv Machine Learning
Sep 14

AnyView: Synthesizing Any Novel View in Dynamic Scenes

AnyView is a diffusion-based video generation framework designed for dynamic view synthesis, requiring minimal inductive biases or geometric assumptions. It trains a generalist spatiotemporal implicit representation using diverse data sources—monocular, multi-view static, and multi-view dynamic—to produce zero-shot novel videos from arbitrary camera locations and trajectories. The authors evaluate AnyView on standard benchmarks, introduce a new challenging benchmark called AnyViewBench for extreme dynamic view synthesis, and demonstrate that AnyView outperforms baselines in maintaining realistic, plausible, and spatiotemporally consistent videos across diverse real-world scenarios.

By Basile Van Hoorick, Dian Chen, Shun Iwase, Pavel Tokmakov, Muhammad Zubair Irshad, Igor Vasiljevic, Swati Gupta, Fangzhou Cheng, Sergey Zakharov, Vitor Campagnolo Guizilini