← Back to all news
arXiv Machine Learning By Farhad Pashakhanloo, Jacob A. Zavatone-Veth

Stimulus symmetries can confound representational similarity analyses

Read the original on arXiv Machine Learning →

The paper investigates how symmetries in network inputs can mislead representational similarity analyses (RSAs). It shows that functionally equivalent representations, arising from stimulus symmetries, can produce distinct RSMs that depict different representational geometries—from disentangled to maximally mixed. The authors demonstrate that training dynamics such as stochastic gradient descent or energetic regularization can yield sparse, drifting codes, causing RSMs to drift over time, and that these effects also appear in networks trained on image data with latent symmetries.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 2

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

The paper investigates the relationship between representation and function in neural networks, using analytical two‑layer linear models and simulations of nonlinear networks. It demonstrates that functional similarity and representational similarity can be dissociated: networks may share representations without sharing functions, and vice versa. The study also finds that robustness to input noise or generalization error does not constrain representations, whereas robustness to parameter noise forces networks to adopt task‑specific representations, indicating that representational alignment reflects computational advantages beyond mere functional alignment.

By Lukas Braun, Erin Grant, Andrew M. Saxe
safety
More like this →
arXiv Machine Learning
Oct 2

Parameter symmetries determine representational geometry in overparameterized nonlinear networks

The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.

By Marvin Theiss, Lukas Braun, Andrew M. Saxe, Erin Grant
More like this →
arXiv Machine Learning
Jun 24

Similarity of Neural Network Representations in Superposition

arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.

By Sunny Liu, Habon Issa, Andr\'e Longon, Liv Gorton, Meenakshi Khosla, Alex Williams, David Klindt
llmssafety
More like this →