arXiv AI
World Modeling through Spectral Alignment
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 3
Learning Commute-Time-Preserving World Models for Planning
The paper introduces Commute-Time-Preserving World Models (CTWMs), which learn latent representations that reflect commute-times in an environment by using a latent displacement predictor and a log-determinant regularizer. This approach addresses the issue that existing self-supervised methods degrade the necessary eigenvalue-dependent scaling for accurate commute-time representation. In experiments, CTWMs outperform the task-agnostic baseline LeWM on several continuous goal-reaching benchmarks while using only half the parameters.
arXiv Machine Learning
Sep 7
Spectral-Target Physical Latent Structuring for JEPA-Style World Models
The paper introduces a lightweight Fourier auxiliary head to enforce physically-informed structuring of latent states in JEPA-style world models, addressing a newly identified failure mode called physical representation laziness that hampers planning in dynamic environments. Experiments show that this auxiliary supervision improves planning success rates, enhances latent space correlations with key physical properties, and boosts data efficiency, even when the baseline model does not exhibit laziness.
arXiv Computer Vision
Sep 28
WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving
WALT introduces a method to align latent trajectories with pretrained driving world models, creating a compact generative trajectory space that preserves action-relevant semantics without altering the original model. The approach uses a dual-branch autoencoder to map raw waypoints into this latent space and transfers visual world knowledge into trajectory representations. Experiments on NAVSIM benchmarks show modest performance gains and a 30.5% reduction in planner FLOPs, indicating that maintaining world representations while extracting action-relevant information can improve trajectory planning efficiency.
arXiv AI
Jun 19
Sensorimotor World Models: Perception for Action via Inverse Dynamics
arXiv:2606. 20104v1 Announce Type: cross Abstract: Perception for action suggests that representations of the world should be shaped not by visual fidelity alone, but by their relevance for actions.
arXiv AI
Sep 4
Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning
The paper presents an end‑to‑end JEPA world model that enhances latent prediction with inverse dynamics and state alignment to improve goal‑conditioned robotic planning. By preventing latent collapse and grounding representations in physical configuration, the model achieves top success rates on tasks such as TwoRoom, PushT, and OGBench‑Cube, outperforming the baseline LeWorldModel. Ablation studies confirm that state alignment consistently boosts planning success over inverse dynamics alone across all four benchmark tasks.
arXiv Machine Learning
Jun 5
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
arXiv:2603. 19312v3 Announce Type: replace Abstract: Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse.