arXiv AI
Jun 30
Flow Matching in Feature Space for Stochastic World Modeling
arXiv:2606. 29059v1 Announce Type: cross Abstract: World modeling requires forecasting uncertain futures while preserving information useful for downstream perception.
IntactWorld: Joint World Modeling with Intact Features
IntactWorld introduces a Joint World Modeling Architecture that preserves uncompressed Intact Features to better capture real‑world logic in video generation. By predicting clean intermediate features instead of flow velocity in a high‑dimensional space, it avoids manifold gaps and reduces computational overhead through a Full‑to‑Compact Training Paradigm that replaces raw features with refined CLS tokens. The approach cuts spatial memory usage by 11.4% and inference latency by 43.8%, achieving a 2.46‑point improvement on the VBench 2.0 benchmark.
Related stories
arXiv AI
Jun 19
Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think
arXiv:2606. 20246v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference.
arXiv Computer Vision
Sep 3
SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models
SolarWM is an open foundation for building interactive video world models, offering a reconfigurable multi‑source data engine that unifies 1.43 million clips from 10 datasets into a consistent, frame‑aligned format. It provides a backbone‑native adaptation framework that preserves native representations of models ranging from 5 B to 33 B parameters, and a three‑stage training recipe combining bidirectional adaptation, teacher‑forced autoregressive initialization, and distribution‑matching distillation. The resulting causal models can interact in real‑time over rollouts from minutes to hours, trained only on 5‑second sequences, and the project releases data, pipeline, recipes, weights, and framework for reproducible research.
arXiv Computer Vision
Aug 24
Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understanding
arXiv:2608.21136v1 Announce Type: new Abstract: Recently, open-vocabulary zero-shot 3D scene understanding using vision foundation models has emerged as a promising alternative to data-intensive supe...
arXiv Computer Vision
3d ago
ControlFlow3D: Distilling Multi-View Knowledge into Latent Flow Matching for Point Cloud Upsampling
ControlFlow3D is a framework that upsamples point clouds by distilling knowledge from pretrained foundation models into a lightweight generation network. It performs conditional flow matching in a latent space defined by a frozen 3D encoder, using unsupervised soft KMeans to cluster multi‑view image tokens and 3D latent tokens into shared prototypes that guide the flow trajectory. A Mamba‑based geometry decoder then maps latent features to dense coordinates with linear complexity, and the authors introduce ShapeNetPU, a 35K‑object multi‑modal benchmark 31 times larger than previous datasets.
arXiv Machine Learning
Jul 23
WorldPack: Dynamic Frame Compression for Long-context Video World Modeling
arXiv:2512. 02473v2 Announce Type: replace-cross Abstract: Video world models have attracted significant attention for their ability to produce high-fidelity future visual observations conditioned on past observations and navigation actions.
arXiv Machine Learning
Jun 5
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
arXiv:2603. 19312v3 Announce Type: replace Abstract: Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse.
arXiv Computer Vision
Sep 1
Off-Manifold Refinement: Guiding Video Generators with a Frozen World Model
arXiv:2608.29904v1 Announce Type: new Abstract: Modern video generators routinely fail at physical dynamics: objects float, trajectories violate gravity, contacts vanish. Standard denoising and flow-...
arXiv Computer Vision
Oct 2
Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models
arXiv:2610.01942v1 Announce Type: new Abstract: Predicting the future evolution of a scene is a fundamental capability for world modeling. Recent work has shown that operating in the feature space of...
arXiv Computer Vision
Oct 2
PAGER: Partial-to-global Alignment via Geometric and Relational Distillation
PAGER is a label‑free adaptation method that aligns partial, viewpoint‑dependent 3D observations with a frozen global semantic space. It uses matched‑point feature alignment and relational supervision to anchor partial features to their global counterparts while preserving similarity structure, all without altering the pretrained encoder or global probe. Experiments show PAGER outperforms label‑supervised PEFT on Sonata and Concerto, and achieves superior zero‑shot transfer from ScanNet to ScanNet++ compared to fully fine‑tuned Sonata.
Hugging Face Trending Papers
Jul 21
Latent Riemannian Flow Matching for Geometry-Grounded 3D Foundation Models
Geometric foundation models, such as the Visual Geometry Grounded Transformer (VGGT), provide strong 3D priors from unposed images. However, such models operate purely in a feed-forward, deterministic regime, \ie~they cannot generate plausible geometry beyond what the input views directly support.
arXiv Computer Vision
Sep 28
WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving
WALT introduces a method to align latent trajectories with pretrained driving world models, creating a compact generative trajectory space that preserves action-relevant semantics without altering the original model. The approach uses a dual-branch autoencoder to map raw waypoints into this latent space and transfers visual world knowledge into trajectory representations. Experiments on NAVSIM benchmarks show modest performance gains and a 30.5% reduction in planner FLOPs, indicating that maintaining world representations while extracting action-relevant information can improve trajectory planning efficiency.