arXiv:2608. 16664v1 Announce Type: cross Abstract: We study the Random Quadratic Form (RQF) on a sphere in the presence of random Brownian forcing.
By Anna Shalova
The paper investigates the discrete-time self‑attention dynamics of transformers, providing a full classification of equilibria and their stability for both single‑head and multi‑head cases. It shows that the main attractor is a consensus equilibrium aligned with the principal eigenvector of the value matrix, while locally asymptotically stable bipartite equilibria also exist in the single‑head scenario. The theoretical findings are corroborated by examining learned weight matrices from an actual transformer.
By Claudio Altafini
arXiv:2501. 18322v2 Announce Type: replace Abstract: Transformers, which are state-of-the-art in most machine learning tasks, represent the data as sequences of vectors called tokens.
By Val\'erie Castin, Pierre Ablin, Jos\'e Antonio Carrillo, Gabriel Peyr\'e
arXiv:2602. 14885v2 Announce Type: replace-cross Abstract: Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that restricts network dynamics to gradient-like flows.
By Ram\'on Nartallo-Kaluarachchi, Renaud Lambiotte, Alain Goriely
arXiv:2608. 08922v1 Announce Type: cross Abstract: Transformer layers generate state-dependent interaction networks: token representations determine the attention matrix, which in turn updates the representations.
By Qucheng Gao, Zuyi Yang, Xiao Chen
arXiv:2608. 14638v1 Announce Type: new Abstract: In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points.
By Leonid Berlyand, Roman Sarapin, Yitzchak Shmalo, Victor Slavin, Sasha Sodin