OccJEPA treats occlusion as masking for amodal video segmentation by dropping tokens outside an object's visible mask and using a pre‑trained V‑JEPA 2.1 predictor to fill in the missing features, followed by a lightweight head that decodes these into complete masks. The resulting deterministic model is 33.5× faster than diffusion‑based state‑of‑the‑art methods and achieves higher accuracy on SAIL‑VOS, TAO‑Amodal (including a held‑out test split), and Kubric. Experiments show that the performance gains stem from the predictor’s pre‑trained weights rather than its size, with models ranging from 87 M to 1.85 B parameters.
By Genpei Zhang
arXiv:2609.37378v1 Announce Type: cross
Abstract: Latent world models are trained to predict what happens next, so nothing in their objective separates what an action caused from what merely co-occur...
By Hossein Resani, Javen Qinfeng Shi
arXiv:2609.09528v1 Announce Type: new
Abstract: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: h...
By Dhairya Bhatia, Bishoy Galoaa, Oliver Fritsche, Shahid Kamal, Muhammad Obaidullah Abdul Salam, Umer Saleem, Om Rastogi, Frania Felix Chettiar, Nesli Erdogmus, Sarah Ostadabbas
LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.
By Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, Florian Buettner
CALIPER is a new benchmark that tests whether pretrained visual encoders can infer physical properties such as mass and friction from images. The test involves striking an object twice at known speeds, showing a third strike only up to contact, and asking a linear readout on frozen features to predict how far the object slides. Results show that in clean, fixed‑camera scenes all representations perform similarly, but when camera, lighting, and clutter are varied, only encoders that truly infer physics—like V‑JEPA 2—maintain performance, while random or raw pixel representations fail.
By Aman Mehta, Riya Baviskar
arXiv:2608.30692v1 Announce Type: new
Abstract: Video world models are increasingly used as simulators, yet visual fidelity alone does not show that a model maintains the hidden state of the world. W...
By Joonghyuk Shin, Yicong Hong, Jaesik Park, Xun Huang
arXiv:2609.25541v1 Announce Type: new
Abstract: Tabular foundation models learn to predict cell values in context, whereas world-model self-supervision asks for prediction in representation space (Le...
By Mingyu Jeon, Suwan Cho, Jae Young Suh
The paper challenges the common practice of selecting latent world models based on one‑step prediction loss, arguing that this metric poorly predicts planning performance. Using a frozen‑decoder protocol, the authors decompose one‑step error into encoder‑inherent and predictor‑added components and evaluate goal‑conditioned planning across three continuous‑control environments. Their experiments show that improvements in one‑step fidelity do not correlate with planning success, with correlation coefficients near zero and inconsistent exchange rates across tasks.
By Joshua Cheung, Zhang He
TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation proposes a lightweight mechanism that learns per-slot, per-frame activation scores to control slot lifecycle in unsupervised video object-centric learning. By gating slot updates and decoder participation when objects are absent or occluded, TSA reduces state drift and reconstruction interference, improving object decomposition and temporal identity preservation. The method is evaluated on MOVi-C, MOVi-E, YTVIS-HQ, and OVIS benchmarks, showing significant gains, especially on long, heavily occluded videos.
By Duc Nguyen, Sieu Tran, Hao Vo, Khoa Vo, Duy Minh Ho Nguyen, Nghi D. Q. Bui, Anh Nguyen, Long Mai, Ngan Le
The paper introduces SLIM, a pushing benchmark that tests language‑based goal specification for latent world models. It shows that a vanilla LeWM fails almost entirely due to an action‑insensitive encoder, but adding an inverse‑dynamics auxiliary loss restores encoder sensitivity and dramatically improves success rates. With the repaired latent, a lightweight language‑goal head can plan from natural language sentences, achieving high performance on navigation and pushing tasks without retraining the world model.
By Florian Strohm, Patrick Wagner, Jannik Schwab, Marco Huber
arXiv:2605.15618v2 Announce Type: replace-cross
Abstract: Self-supervised video models are increasingly framed as world models, yet they are still evaluated almost entirely on clean video and reporte...
By Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam, James Bailey, Naveed Akhtar
arXiv:2610.16365v1 Announce Type: new
Abstract: Video world models must generate physically plausible futures, not just realistic frames. Existing approaches typically use semantic, physical, or pred...
By Shinseong Kim, Jaeseok Jeong, Jianlong Fu, Youngjung Uh