The paper investigates associative memory in a bipartite Hopfield–Krotov architecture, termed class H, where hidden neurons serve as the retrieval order parameter. Using the replica method, it derives replica‑symmetric phase diagrams and closed‑form capacities for polynomial load, showing that crosstalk statistics are similar for Ising and spherical visible neurons. With a softmax hidden layer, the load becomes exponential, mapping the thermodynamics onto a random‑energy‑model that exhibits paramagnetic, condensed, and frozen phases, and revealing that heating destabilizes retrieval through quantized attention reassignments while Gaussian patterns remain metastable at all loads.
By Toshihiro Ota, Masato Taki
arXiv:2606. 12059v1 Announce Type: new Abstract: We address transformer attention on energy-constrained physical substrates.
By Fabio Pasqualetti, Taosha Guo
arXiv:2512. 10656v3 Announce Type: replace Abstract: As context windows in large language models continue to expand, it is essential to characterize how attention behaves at extreme sequence lengths.
By L\'ea Bohbot, Cyril Letrouit, Gabriel Peyr\'e, Fran\c{c}ois-Xavier Vialard
arXiv:2607. 11950v1 Announce Type: cross Abstract: Brain field potentials are scale-free: their power spectra follow a $1/f^{\beta}$ law whose aperiodic exponent $\beta$ tracks cortical state, and sleep depth in particular is a shift in $\beta$.
By Dibakar Sigdel
arXiv:2609.37535v1 Announce Type: new
Abstract: In the softmax output layer, a rare token receives a small positive logit gradient on most steps and a much larger negative gradient on the few steps w...
By Sangsidhya Kar
arXiv:2609.37745v1 Announce Type: new
Abstract: Neural scaling, in which loss falls as a power law with training, is central to large language models, and one recent proposal is that a $1/3$ exponent...
By Hyunseok Lee, Mihir Basil, Yizhou Liu, Jeff Gore
arXiv:2611.06055v1 Announce Type: cross
Abstract: Attention is central to modern AI models for language, vision, and scientific modeling. However, direct evaluation compares every query with every ke...
By Zixuan Huang (State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China, School of Physics, Peking University, Beijing, China), Guozhang Chen (State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China, National Engineering Research Center of Visual Technology, School of Computer Science, Peking University, Beijing, China)
arXiv:2608. 19171v1 Announce Type: new Abstract: Deep models for irregularly-sampled time series answer queries at arbitrary continuous timestamps, yet report nothing about how far each answer should be trusted.
By Sotirios P. Chatzis, Loukas Papadoulas
Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the property of statistical ada...
arXiv:2605.12049v2 Announce Type: replace-cross
Abstract: Cortical neurons are complex, multi-timescale processors wired into recurrent circuits, shaped by long evolutionary pressure under stringent...
By Aaron Spieler, Georg Martius, Anna Levina
arXiv:2608. 13335v1 Announce Type: new Abstract: Neural networks trained by gradient descent on a smooth cost function can nevertheless learn in steps: the cost holds on long plateaus and then drops abruptly.
By Liu Ziyin, Yizhou Xu, Tomaso Poggio, Isaac Chuang
arXiv:2605.10795v2 Announce Type: replace-cross
Abstract: Large language models excel at factual recall, yet the fundamental limits of storing input--output associations with neural networks remain u...
By Alessio Giorlandino, Sebastian Goldt, Antoine Maillard