arXiv Machine Learning By Akarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanley

Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis

Read the original on arXiv Machine Learning →

The paper challenges the assumption that improved performance in deep learning automatically leads to better internal representations. By comparing networks evolved through open‑ended search with those trained by stochastic gradient descent on a simple image‑generation task, the authors find that both produce identical outputs but differ markedly in their internal structure. The SGD‑trained models exhibit a fractured entangled representation (FER), whereas the evolved models approach a unified factored representation (UFR).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Oct 28

FlowGenesis: Representation-Driven Self-Organization of Neural Architectures

FlowGenesis is a framework that lets neural networks self‑organize their architecture during training by adding or removing neurons based on a local statistic called Representation Flux. Neurons with high Flux are split to preserve function, while low‑Flux, low‑activity neurons are pruned. Experiments on image and text classification show that networks starting from different sizes converge to similar neuron counts and performance, and the method also improves continual learning by reducing parameters for new tasks without sacrificing accuracy.

By Maksim A Kazanskii
arXiv AI
Sep 15

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.

By Safa Hamreras, Sukhbinder Singh, Rom\'an Or\'us
Hugging Face Trending Papers
Aug 4

DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformers

Recent advances in Diffusion Transformers (DiTs) have enabled remarkable progress in visual synthesis, benefiting from their superior scalability. To facilitate DiTs' capability of capturing meaningful internal representations, recent works such as REPA incorporate external pretrained encoders for representation alignment.