The Information Geometry of Representation Formation
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
The Flow has not summarised this story yet — read it at arXiv Statistics ML.
arXiv:2609.36081v1 Announce Type: new Abstract: Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure th...
The paper investigates how encouraging action predictability in latent world models can preserve intrinsic dynamics, which are crucial for planning. By deriving a theoretical lower bound and conducting empirical tests, the authors show that action predictability prevents the encoder from merging distinct action outcomes, even when visual appearance varies. Using a method called Concat IDM, they achieve a substantial increase in planning success on the Push‑T benchmark, from 9.1% to 77.2%, demonstrating the practical benefit of this inductive bias.
arXiv:2609.05575v1 Announce Type: new Abstract: Understanding how concepts are encoded in the internal representations of machine learning models is a central problem in mechanistic interpretability,...
Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions, raising a basic question: what local metric bes...
arXiv:2607. 02386v1 Announce Type: cross Abstract: While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout training remains insufficiently understood.
arXiv:2606. 00124v1 Announce Type: cross Abstract: Positional embeddings (PEs) in Vision Transformers (ViTs) are known to impact performance and robustness, but their role in shaping internal spatial representations is not well understood.