The paper introduces DSReg, a method that provably recovers individual world latent variables without relying on reconstruction, decoders, or labels. It leverages a Structural Diversity condition—distinct dependency footprints of different latents—to achieve full identifiability up to signed permutation. DSReg can be applied post hoc to any linearly identified representation, preserving dense prediction while improving latent recovery across synthetic and real-world experiments.
By Yujia Zheng, David Klindt, Randall Balestriero, Bernhard Sch\"olkopf
The paper argues that the linear representation hypothesis—where neural activations encode concepts as linear mixtures—breaks down under superposition, causing linear decision boundaries to lose linearity after projection. Classical sparse coding with per‑sample inference can recover latent factors, but sparse autoencoders (SAEs) amortise this inference into a fixed encoder, creating a systematic gap that persists across training sizes, latent dimensions, and sparsity levels. Experiments show that the failure of SAEs under out‑of‑distribution compositional shifts stems from poor dictionary learning rather than inference, and that a good dictionary can solve the problem at all tested scales, highlighting scalable dictionary learning as the key open challenge.
By Vit\'oria Barin-Pacela, Shruti Joshi, Isabela Camacho, Simon Lacoste-Julien, David Klindt
arXiv:2606. 28854v1 Announce Type: cross Abstract: The common factor analytic model is related to Helmholtz and Boltzmann machines, can be conceived as a linear autoencoder, or can be thought of as a single-hidden-layer generative neural network.
By Carel F. W. Peeters
The paper introduces a lightweight, observation-conditioned latent energy prior to improve inference for frozen implicit neural representation (INR) decoders when only sparse off‑grid signed distance function (SDF) samples are available. By standardizing latent codes based on a permutation‑invariant encoding of the sparse observations and combining this energy with a validation‑selected L2 prior, the method consistently outperforms baseline L2 and a six‑component Gaussian mixture model prior on both a controlled cell‑nucleus SDF dataset and a MedShapeNet‑derived SDF completion dataset, especially in the sparsest regimes. Ablation studies confirm that the energy term’s contribution is specific to the observed context rather than generic.
whyItMatters":"The approach demonstrates that pretrained INR decoders can become more observation‑aware without retraining, improving shape completion accuracy in data‑sparse scenarios."
By Paul B\"uschl, Ezequiel de la Rosa, Julia Wolleb, Julian McGinnis, C\'esar Nombela-Arrieta, Bjoern Menze
arXiv:2503. 21796v2 Announce Type: replace-cross Abstract: Self-supervised learning has become an increasingly important paradigm in the domain of machine intelligence.
By Alexander Ororbia, Karl Friston, Rajesh P. N. Rao
arXiv:2607. 16295v1 Announce Type: cross Abstract: Mechanistic interpretability has made significant strides in understanding neural network representations, with sparse dictionary learning (SDL) methods, most prominently sparse autoencoders, as a central paradigm.
By Yiming Tang, Qinglin Qi, Zhaoqian Yao, Harshvardhan Saini, Dianbo Liu
arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.
By Ege Erdogan, Ana Lucic
MOSAIC is a sparse temporal variational autoencoder that merges causal representation learning with support recovery to identify interpretable latent modules in scientific time series. It leverages regime‑conditioned temporal variation to discover latent variables and uses an additive decoder to recover a sparse set of observations linked to each latent, providing module‑level interpretability. The method offers theoretical guarantees for ANOVA main‑effect identifiability under smooth mixing functions and demonstrates empirical success across diverse domains such as RNA dynamics, solar wind, ENSO climate, industrial process control, and synthetic tokamak data.
By Shicheng Fan, Nour Elhendawy, Jianle Sun, Ke Fang, Kun Zhang, Yihang Wang, Lu Cheng
AdaKerNet is a task‑adaptive neural kernel decoder that operates on frozen multimodal representations from large foundation models, without requiring access to the models’ parameters. It learns Lipschitz‑controlled multimodal features, a reference kernel providing a soft structural prior, and a lightweight nonlinear predictor that deforms this structure. Experiments on four multimodal large language models and diverse input modalities show consistent improvements over baseline decoders, achieving up to 41% error reduction in scarce‑label settings.
By Konstantinos D. Polyzos, Eleni Oikonomou, Tara Javidi
arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.
By Sunny Liu, Habon Issa, Andr\'e Longon, Liv Gorton, Meenakshi Khosla, Alex Williams, David Klindt
arXiv:2606. 11722v1 Announce Type: cross Abstract: Finding interpretable directions in language-model representations is critical for understanding and controlling model behavior.
By Sida Liu, Feijiang Han
arXiv:2602. 10680v2 Announce Type: replace-cross Abstract: Many real-world datasets contain hidden structure that cannot be detected by simple linear correlations between input features.
By Vicente Conde Mendes, Lorenzo Bardone, C\'edric Koller, Jorge Medina Moreira, Vittorio Erba, Emanuele Troiani, Lenka Zdeborov\'a