arXiv Machine Learning

Discovering Nonlinear Static Relationships in Unlabeled Dataset using Autoencoder with Ordered Variance

arXiv:2402. 14031v2 Announce Type: replace-cross Abstract: This paper presents an autoencoder with ordered variance (AEO), in which the conventional reconstruction loss is augmented by a variance-based regularization term that promotes an ordered structure within the latent space.

arXiv Machine Learning
Aug 11

A solvable high-dimensional model where nonlinear autoencoders learn structure invisible to PCA while test loss misaligns with generalization

arXiv:2602. 10680v2 Announce Type: replace-cross Abstract: Many real-world datasets contain hidden structure that cannot be detected by simple linear correlations between input features.

By Vicente Conde Mendes, Lorenzo Bardone, C\'edric Koller, Jorge Medina Moreira, Vittorio Erba, Emanuele Troiani, Lenka Zdeborov\'a
arXiv Machine Learning
Aug 24

Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data

The paper introduces Conditional-Independence-Regularized Distributional Autoencoders, a framework for learning low-dimensional representations of mixed-type data that includes both numerical and categorical variables. It uses an energy-score objective for numerical variables, a likelihood objective for categorical variables, and an auxiliary conditional independence regularization term to capture dependencies between variable types. The authors provide theoretical analysis and demonstrate that the method improves categorical distribution recovery, achieves competitive overall conditional distribution recovery, and preserves mixed-type dependence structure on synthetic and real-world datasets.

By Siyuan Tang, Gongjun Xu, Ji Zhu
arXiv Machine Learning
1d ago

Deep Symmetric Autoencoders from the Eckart-Young-Schmidt Perspective

The paper presents a theoretical analysis of symmetric autoencoders, a class of deep learning architectures frequently used in machine learning tasks. It distinguishes between different symmetric designs and shows that the reconstruction error of orthonormal symmetric autoencoders can be interpreted via the Eckart‑Young‑Schmidt theorem. Building on this insight, the authors propose an EYS‑based initialization strategy using repeated SVD, and validate its effectiveness through numerical experiments comparing it to conventional deep autoencoders.

By Simone Brivio, Nicola Rares Franco