arXiv:2608.23565v1 Announce Type: new Abstract: An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: contro...
An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounde...
SolarWM is an open foundation for building interactive video world models, offering a reconfigurable multi‑source data engine that unifies 1.43 million clips from 10 datasets into a consistent, frame‑aligned format. It provides a backbone‑native adaptation framework that preserves native representations of models ranging from 5 B to 33 B parameters, and a three‑stage training recipe combining bidirectional adaptation, teacher‑forced autoregressive initialization, and distribution‑matching distillation. The resulting causal models can interact in real‑time over rollouts from minutes to hours, trained only on 5‑second sequences, and the project releases data, pipeline, recipes, weights, and framework for reproducible research.
Manifold4D introduces a new denoising strategy for video re‑shooting that injects a rendered point‑cloud directly into the initial noise manifold, eliminating the need for the render to be an explicit conditioning stream during denoising. This approach allows the network to rely solely on the source video as a visual condition, improving camera‑control accuracy on the DAVIS‑Traj benchmark and Vista4D set, with significant reductions in rotation and translation errors while maintaining video fidelity. User studies confirm enhanced trajectory following and dynamic consistency, especially for large yaw amplitudes and even when the render is corrupted.
arXiv:2608. 07420v1 Announce Type: new Abstract: World models are expected to support imagination over extended temporal horizons, yet most are still trained through local few-step prediction objectives and deployed by recursively rolling out their own predictions.
arXiv:2606. 10135v1 Announce Type: cross Abstract: Transitioning bidirectional video diffusion models into an autoregressive paradigm improves the interactivity of video world models, but existing causal pipelines need many stages (control fine-tuning, autoregressive training, causal initialization, few-step distillation) and still trail bidirectional models in quality due to error accumulation.
arXiv:2606. 28128v1 Announce Type: cross Abstract: Video generation models have emerged as a promising paradigm for embodied world simulation.
arXiv:2601. 14430v2 Announce Type: replace-cross Abstract: Controlling generative models is computationally expensive.
arXiv:2607. 18367v1 Announce Type: new Abstract: Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interactive environments from user inputs instantly.
arXiv:2606. 11075v1 Announce Type: new Abstract: Aligning text-to-image flow matching models with human preferences via direct reward backpropagation is sample-efficient but hampered by two well-known pathologies: activations cannot be stored across the full sampling trajectory at modern model scale, and chained Jacobian products across steps inflate the reward gradient as it travels back to early indices.
PhysMLLMs introduces physics-inspired spatial continuity priors into video multimodal large language models to address spatio‑temporal inconsistencies such as jitter, drift, and identity switches. The method, called Global Representation Prior Alignment (REPA‑Global), distills global visual representations from a frozen DINOv2 teacher during training, aligning student representations without affecting inference speed. Experiments on multiple video benchmarks show improved segmentation mask quality and cross‑frame consistency, especially for challenging scenarios involving small targets, fast motion, occlusion, and distractors, while maintaining comparable performance on single‑frame image segmentation and general VLM tasks.
arXiv:2603. 14294v3 Announce Type: replace-cross Abstract: Do video diffusion models encode signals predictive of physical plausibility?