The paper introduces ALPS, a pretrained visual state encoder that integrates physical-state information, native language alignment, and pixel reconstruction into a single representation. Trained from scratch on still images and text, ALPS uses language, reconstruction, and spatial correspondence objectives, followed by cross-view self‑distillation. In evaluations across fifteen encoders, ALPS achieves the lowest aggregate LIBERO state‑probe error among ViT‑B models, while also delivering strong language alignment, accurate pixel reconstruction, and high policy success in downstream world‑model tasks.
arXiv:2606. 04433v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used in multi-image, multi-turn agentic settings where decisions depend on visual changes.
arXiv:2610.19825v1 Announce Type: cross Abstract: Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynam...
arXiv:2605.15195v2 Announce Type: replace Abstract: Recent feed-forward reconstruction models, such as VGGT, have proven competitive with traditional optimization-based reconstructors while also prov...
arXiv:2609.37250v1 Announce Type: cross Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrai...
arXiv:2609.34826v2 Announce Type: replace Abstract: Humans often solve spatial problems by mentally simulating visual transformations. In contrast, conventional vision-language models (VLMs) reason p...
arXiv:2610.30724v2 Announce Type: replace-cross Abstract: Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pre...
Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.
WorldCast is a benchmark that tests multimodal generative models on multi-step world state prediction, requiring them to forecast successive future states from multimodal observations and diverse references. The study evaluates 12 models across 320 manually designed cases in eight categories, revealing that advanced models like GPT-Image-2.5 achieve high scores while open-source models lag behind, especially on complex context and reasoning tasks. The results underscore the importance of integrating dynamic planning, reasoning, and unified multimodal capabilities for realistic world simulation.
WorldAgen is a unified framework that jointly learns world modeling and action prediction using a shared Transformer backbone with two specialized heads. It introduces a Mixed Unidirectional Attention Mask to separate the world model and agent model, and enables Test-Time Training (TTT) by sampling exploratory actions and updating the world model with real state transitions. Experiments on CALVIN and LIBERO show that WorldAgen matches or surpasses state‑of‑the‑art methods, especially when TTT is applied to a few samples.
arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...
arXiv:2608. 08676v1 Announce Type: cross Abstract: Semantic vision encoders have become a central visual interface for multimodal understanding and semantic conditioning in image generation.
arXiv:2605. 16713v2 Announce Type: replace-cross Abstract: Modern Vision-Language Models (VLMs) achieve strong semantic recognition, yet remain brittle on elementary spatial relations such as left of, on, behind, and between.