Long-range learning is hard for recurrent networks trained with stochastic gradient descent, because the influence of a past input fades with the lag $\ell$, and if it fades too fast the dependence cannot be learned from finite data. This fade is captured by an envelope $f(\ell)$.
arXiv:2606. 29519v1 Announce Type: new Abstract: Long-range learning is hard for recurrent networks trained with stochastic gradient descent, because the influence of a past input fades with the lag $\ell$, and if it fades too fast the dependence cannot be learned from finite data.
By Lorenzo Livi
arXiv:2606. 30789v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has become a standard tool for improving the reasoning ability of large language models, yet its training dynamics are still described empirically: reward trajectories are fit with low-parameter functional forms whose constants carry no mechanistic meaning, and hyperparameter choices remain a matter of trial and error.
By Rajat Ghosh, Datta Nimmaturi, Aryan Singhal, Vaishnavi Bhargava, Henry Wong, Johnu George, Debojyoti Dutta
The paper introduces a world model that learns to predict the evolution of physical systems while respecting key physical principles. By hard‑coding a general structure—generating dynamics from the gradient of a learned energy via a fixed reversible operator and imposing constraints on energy, dissipation, and interventions—the model achieves second‑law compatible dissipation, accurate responses to parameter changes, long‑term stability, and robustness to disturbances. Experiments on an electromagnetic cavity, a particle‑in‑cell grid, and shallow‑water fluid demonstrate that the model can recover accurate constitutive functions, distinguish conserving from dissipating regimes, and transfer learned physics to unseen conditions, outperforming unconstrained models.
By Yufeng Wang, Parivesh Priye, Lu Wei, Haibin Ling
arXiv:2507.10383v5 Announce Type: replace-cross
Abstract: Recurrent neural networks are canonical models of biological memory. In these models, memories are represented by distributed patterns of neu...
By Uri Cohen, M\'at\'e Lengyel
arXiv:2606. 18080v1 Announce Type: new Abstract: Gradient descent in deep learning may operate at the edge of stability (EoS), a regime in which the largest eigenvalue of the loss Hessian hovers near the stability threshold $2/\eta$, where $\eta$ is the learning rate.
By Pierre Marion