arXiv:2609.36081v1 Announce Type: new
Abstract: Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure th...
By Yuantao Deng, Jinnuo Liu, Kaizhen Tan, Yuchen Liu
The paper investigates how encouraging action predictability in latent world models can preserve intrinsic dynamics, which are crucial for planning. By deriving a theoretical lower bound and conducting empirical tests, the authors show that action predictability prevents the encoder from merging distinct action outcomes, even when visual appearance varies. Using a method called Concat IDM, they achieve a substantial increase in planning success on the Push‑T benchmark, from 9.1% to 77.2%, demonstrating the practical benefit of this inductive bias.
By Jiamin Shi, Yuanhong Zhang, Zhiyang Liu, Jiawei Sun, Jingwen Fu
arXiv:2609.05575v1 Announce Type: new
Abstract: Understanding how concepts are encoded in the internal representations of machine learning models is a central problem in mechanistic interpretability,...
By Yiming Tang, Harshvardhan Saini, Samyak Jha, Huaming Chen, Xufeng Duan, Dianbo Liu
Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions, raising a basic question: what local metric bes...
arXiv:2607. 02386v1 Announce Type: cross Abstract: While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout training remains insufficiently understood.
By Kaustubh Kapil, Kishor P. Upla
arXiv:2606. 00124v1 Announce Type: cross Abstract: Positional embeddings (PEs) in Vision Transformers (ViTs) are known to impact performance and robustness, but their role in shaping internal spatial representations is not well understood.
By Mahmoud Mannes
arXiv:2609.36458v1 Announce Type: new
Abstract: Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions...
By Abdullah All Tanvir, Xin Zhong
arXiv:2606. 17889v1 Announce Type: cross Abstract: Compositional learning systems must balance plasticity, the ability to acquire new knowledge, with stability, the preservation of previously learned components, especially when tasks share structure and risk interference.
By Kathrin Korte, Christian Medeiros Adriano, Joachim Winther Pedersen, Eleni Nisioti, Sebastian Risi
The paper introduces Emergent Adaptive Representation Learning (EARL), a framework that identifies two distinct modes of emergent compression in learned representations: state‑space compression, which reflects a representation’s growing predictive power over its own future, and parameter‑space compression, where networks with different micro‑level parameters converge to the same representation. Both modes are formalised using Fisher sensitivities and measured with ensemble‑level information‑theoretic metrics. Experiments on a recurrent reservoir with a trained bottleneck show that parameter‑space compression occurs early in training and is largely task‑independent, while state‑space compression emerges later, coinciding with the network’s learning of temporal task structure.
By Hardik Rajpal, Dan Goodman
arXiv:2609.39195v1 Announce Type: new
Abstract: Spatial intelligence requires maintaining a coherent understanding of the world as the embodied agent moves. Like humans, the agent must use its own mo...
By Shichao Li, Meiqi Wang, Fei Su, Zhicheng Zhao
arXiv:2607. 25531v1 Announce Type: cross Abstract: Contemporary machine learning struggles to learn continually, reuse prior knowledge, and expose a comprehensible internal structure.
By Zeki Doruk Erden
The paper challenges the assumption that improved performance in deep learning automatically leads to better internal representations. By comparing networks evolved through open‑ended search with those trained by stochastic gradient descent on a simple image‑generation task, the authors find that both produce identical outputs but differ markedly in their internal structure. The SGD‑trained models exhibit a fractured entangled representation (FER), whereas the evolved models approach a unified factored representation (UFR).
By Akarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanley