arXiv AI

Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group

arXiv:2606. 03003v1 Announce Type: cross Abstract: A latent world model built from an equivariant encoder $E$ and an equivariant predictor $f$ inherits a provable symmetry of its training loss: when the world's dynamics genuinely carries a group $G$ acting on latents by an orthogonal representation $\rho(g)$, the one-step prediction relMSE is exactly invariant across the whole group, so fitting the dynamics on a restricted slice of orientations mathematically determines it on the entire orbit (j\v{u} y\=i f\v{a}n s\=an).

Hugging Face Trending Papers

Jun 28

Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks

A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation. Adam's per-coordinate preconditioner drifts along each symmetry orbit, which pulls the trajectory off the symmetry quotient where the optimization lives and blurs the singular-learning rate the quotient makes readable.

llms computer-vision

arXiv Machine Learning

Jun 30

Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks

arXiv:2606. 29176v1 Announce Type: new Abstract: A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation.

By Tejas Pradeep Shirodkar

llms computer-vision

arXiv AI

Jul 15

Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter

arXiv:2607. 10203v2 Announce Type: replace-cross Abstract: Adaptive-compute world models -- early-exit or mixture-of-depths predictors that spend variable depth per step -- assume depth buys better predictions and can be routed adaptively.

By Achyuthan Sivasankar

benchmarks

arXiv Machine Learning

Jun 2

Measuring the Symmetry--Data Exchange Rate

arXiv:2606. 01090v1 Announce Type: cross Abstract: Equivariance theory predicts that an architectural symmetry prior reduces sample complexity by a factor of |G|; this is widely cited but rarely measured as a scaling law with controls that separate the prior from its confounds.

By Ahmed M. Adly

safety

arXiv Machine Learning

Jul 3

Certified World Models: Predictability Across Configuration, Horizon, and Resolution

arXiv:2606. 13092v3 Announce Type: replace Abstract: Scale buys interpolation; structure buys certifiable transfer.

By Hongbo Wang

arXiv AI

Jul 14

When Does Depth Survive Composition? Compute--Quality Regimes in Latent World Models

arXiv:2607. 10203v1 Announce Type: cross Abstract: Adaptive-compute world models -- early-exit or mixture-of-depths predictors that spend variable depth per step -- assume depth buys better predictions and can be routed adaptively.

By Achyuthan Sivasankar

benchmarks

arXiv AI

Jun 2

UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures

arXiv:2606. 01443v1 Announce Type: cross Abstract: A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse.

By Triet M. Le

rag

arXiv Machine Learning

Aug 5

ForgettingOT: Certified Speculative Batching from Sinkhorn's Projective Forgetting

arXiv:2607. 24741v2 Announce Type: replace-cross Abstract: Positive two-marginal entropic optimal transport is solved by a nonlinear, positive, order-preserving, homogeneous Sinkhorn map.

By Xinyang Wen

arXiv AI

Aug 10

Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

arXiv:2608. 07436v1 Announce Type: new Abstract: Under the standard split, Muon gets hidden matrices and AdamW embeddings/output head.

By Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau, Anass Belfatmi

llms rag safety

arXiv Machine Learning

Jul 27

Multi-Horizon Consistency as Geometry: When Latent Dynamics Contract, and When They Do Not

arXiv:2607. 21645v1 Announce Type: new Abstract: Multi-horizon latent consistency is a common training knob in video predictors and world models, but practitioners rarely know what it does to transition geometry.

By Kavya Bhand, Aadi Joshi

arXiv AI

Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko

llms fine-tuning

arXiv Machine Learning

Jun 8

High entropy leads to symmetry-equivariant policies in Dec-POMDPs

arXiv:2511. 22581v5 Announce Type: replace Abstract: We prove that in any Dec-POMDP, sufficiently high entropy regularization ensures that the policy gradient flow with tabular softmax parametrization always converges, for any initialization, to the same joint policy, and that this joint policy is equivariant w.

By Johannes Forkel, Constantin Ruhdorfer, Michael Beukman, Andreas Bulling, Jakob Foerster

agents reinforcement-learning benchmarks

Related stories