← Back to all news
arXiv Machine Learning By John J. Vastola

On the stability of learning

Read the original on arXiv Machine Learning →

The paper investigates how learning algorithms behave when gradient signals are noisy, sparse, and intermittently available, as often occurs in biological neural networks. It demonstrates that even if the average learning dynamics converge to the correct optimum, the variance of learning can become asymptotically infinite, leading to instability. The authors analyze several analytically tractable settings—including single-layer and multi-layer linear networks—to derive explicit conditions under which learning remains stable, highlighting that unbiased credit assignment alone is insufficient without sufficiently frequent and informative feedback.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 12

On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective

The paper analyzes Gradient Descent (GD) on a minimal binary classification problem with a two‑neuron ReLU network trained on two instances. It proves that GD converges to an optimal robustness margin—maximizing the distance between the decision boundary and training points—but does so at a very slow rate, specifically θ(1/ln(t)). Empirical simulations confirm this slow convergence across various initializations, and the authors provide a rigorous analysis of GD trajectories across different activation patterns to establish the first explicit lower bound on robustness convergence in a non‑linear model.

By Guy Smorodinsky, Sveta Gimpleson, Itay Safran
safety
More like this →