arXiv:2608. 07349v1 Announce Type: new Abstract: Learning from heterogeneous representations is usually reduced to feature concatenation, which erases which representation produced an error.
arXiv:2608. 03620v1 Announce Type: cross Abstract: Activation patching and weight-space ablation both claim a component is causally responsible for a behavior, yet they act on different objects: one forward pass versus the parameters behind every forward pass.
arXiv:2609.01129v1 Announce Type: new Abstract: We identify a recurrent algebraic regularity in Transformer attention: a sparse subset of effective OV operators $T=OV^\top$ nearly closes under compos...
The paper proposes a new dimension‑reduction strategy for transfer‑operator models of dynamical systems that focuses on scoring the σ‑algebra generated by coordinates rather than the operator’s spectral span. By using a χ²‑divergence criterion between embedded present and future states, the method guarantees that twice the intrinsic system dimension suffices to capture the full operator spectrum, even for systems with weakly interacting components that would otherwise require exponentially many modes. Experiments on benchmark systems show that this algebraic approach recovers masked components missed by rank‑based methods and enables accurate prediction of those components from few labels.
The paper addresses the limitations of spectral dimension reduction for dynamical systems composed of weakly interacting components, where standard rank‑based methods either require exponentially many modes or omit entire components (a phenomenon termed linear masking). It proposes scoring the σ‑algebra generated by coordinates instead of individual modes, using a χ²‑divergence criterion that guarantees an embedding with twice the intrinsic dimension captures the full operator spectrum. Experiments on benchmark systems show that this algebraic approach recovers masked components and enables accurate prediction from few labels, outperforming traditional rank‑based and VAMP methods.
arXiv:2607. 04333v1 Announce Type: new Abstract: Grokking -- generalization arriving long after training-set interpolation -- can be accelerated by structure-agnostic interventions: gradient filtering, weight-norm clamping, geometric penalties on hidden representations.
arXiv:2601. 11618v2 Announce Type: replace-cross Abstract: Geometric Attention (GA) specifies an attention layer by four independent inputs: a finite carrier (what indices are addressable), an evidence-kernel rule (how masked proto-scores and a link induce nonnegative weights), a probe family (which observables are treated as admissible), and an anchor/update rule (which representative kernel is selected and how it is applied).
arXiv:2607. 13749v1 Announce Type: new Abstract: Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately.
arXiv:2609.40063v1 Announce Type: cross Abstract: Low-Rank Adaptive Residual Connections (LARC) give a frozen model a compact numerical state that can learn from feedback. The map $h+BAh$ adds a low-...
arXiv:2609.40127v1 Announce Type: cross Abstract: Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keepin...
Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately. What happens at the extreme of this spectrum, when the architecture's expressible function class collapses to a finite-dimensional algebraic variety?
arXiv:2608.21098v1 Announce Type: new Abstract: Fusing prior knowledge with data-driven learning is attractive where data is scarce, yet no controlled account says when it helps, is redundant, or har...
arXiv:2609.39263v1 Announce Type: new Abstract: A concept subspace's effect on model behavior does not establish how it relates to the output readout. We introduce a two-sided geometric diagnostic th...