The paper investigates when supervised learning can recover the true latent variables that generate data, focusing on two causal settings: generative (labels cause latents) and discriminative (latents cause labels). It shows that if the representation space has at least as many dimensions as the latent space, optimal encoders are linear; otherwise, identifiability fails and encoders become nonlinear. Additional constraints—such as non‑negative activations in the generative case or sparsity and spark conditions in the discriminative case—enable recovery of individual latent variables, linking nonlinear ICA, compressed sensing, and neural network interpretability.
By Junze Zhou, David Klindt
The paper studies when joint-embedding predictive architectures (JEPAs) can recover underlying causal states from high‑dimensional observations. It introduces a latent variable model where observations arise from causal states with action‑conditioned dynamics, and proposes an information‑theoretic objective that maximizes conditional likelihood while preserving state entropy. The authors prove identifiability conditions—particularly sufficient action‑induced variation—and instantiate the objective as an action‑modulated Gaussian additive‑noise model (A‑JEPA), demonstrating theoretical and empirical success in synthetic and visual benchmarks.
By Yuhang Liu, Zhuo Huang, Javen Qinfeng Shi
MOSAIC is a sparse temporal variational autoencoder that merges causal representation learning with support recovery to identify interpretable latent modules in scientific time series. It leverages regime‑conditioned temporal variation to discover latent variables and uses an additive decoder to recover a sparse set of observations linked to each latent, providing module‑level interpretability. The method offers theoretical guarantees for ANOVA main‑effect identifiability under smooth mixing functions and demonstrates empirical success across diverse domains such as RNA dynamics, solar wind, ENSO climate, industrial process control, and synthetic tokamak data.
By Shicheng Fan, Nour Elhendawy, Jianle Sun, Ke Fang, Kun Zhang, Yihang Wang, Lu Cheng
Joint-embedding predictive architectures (JEPAs) learn latent dynamics for planning and avoid representation collapse by matching features to maximum-entropy distributions such as isotropic Gaussians,...
The paper investigates how encouraging action predictability in latent world models can preserve intrinsic dynamics, which are crucial for planning. By deriving a theoretical lower bound and conducting empirical tests, the authors show that action predictability prevents the encoder from merging distinct action outcomes, even when visual appearance varies. Using a method called Concat IDM, they achieve a substantial increase in planning success on the Push‑T benchmark, from 9.1% to 77.2%, demonstrating the practical benefit of this inductive bias.
By Jiamin Shi, Yuanhong Zhang, Zhiyang Liu, Jiawei Sun, Jingwen Fu
arXiv:2606. 17516v1 Announce Type: cross Abstract: Causal discovery from observational data remains challenging due to the need to recover directed structure and latent confounding without interventions.
By Patrick Bl\"obaum, Krishnakumar Balasubramanian, Shiva Prasad Kasiviswanathan
Subspace-Decomposed JEPAs (SD-JEPA) split the latent space of Joint-Embedding Predictive Architectures into two orthogonal subspaces: a low-dimensional progression subspace trained with a cosine-margin triplet loss and a high-dimensional content subspace regularised by SIGReg. The authors prove that the anti-collapse forces act on disjoint coordinates, allowing additive composition rather than competition. SD-JEPA outperforms the LeWM baseline on most control benchmarks and the strongest non-LeWM JEPA baseline on Push‑T, with a subspace-ablation confirming the split as essential. The 1‑D angular progression coordinate serves as a scene-aware compass, advancing with task progress, regressing on backtracking, and relocalising under perturbations to separate surprise from meaning.
By Lucas Thil, Jesse Read, Rim Kaddah, Guillaume Doquet
arXiv:2606. 12362v1 Announce Type: cross Abstract: We study multimodal learning under missing modalities, with particular motivation from bioscience applications in which heterogeneous modalities are often only partially available when decisions need to be made.
By Hui Wang, Tianyu Ren, Joseph Butler, Christopher Baker, Karen Rafferty, Simon McDade
arXiv:2606. 09936v1 Announce Type: cross Abstract: World models are now built on substantially different computational substrates.
By Bhavith Chandra Challagundla, Sanskar Pandey, Param Thakkar, Rishikesh Mallagundla, Yugandhar Reddy Gogireddy, Wenhao Lu, Hindol Roy Choudhury, Shravani Challagundla, Mohamed Deraz Nasr, Spursh Deshpande
SplitJEPA is a new joint embedding predictive architecture that learns a latent state and its invariant and variant components directly in representation space, without reconstructing observations. The method proves that, under stationary Gaussian predictive dynamics and a full‑rank variation condition, it can identify the invariant and variant subspaces up to independent block‑wise isometries. Experiments on synthetic nonlinear systems and robotic manipulation tasks confirm the theoretical guarantees and demonstrate practical benefits for robustness and efficiency.
By Ruijin Hua, Zichuan Liu, Zhuokai Zhao, Yujia Zheng
arXiv:2608.28991v1 Announce Type: cross
Abstract: Causal representation learning (CRL) aims to recover latent causal variables and their structural relations from high-dimensional observations. Exist...
By Haijie Xu, Chen Zhang
arXiv:2608. 20065v1 Announce Type: new Abstract: World models construct latent states that support prediction, planning, and reasoning about an underlying system.
By Taoyong Cui, Pheng Ann Heng, Wanli Ouyang