← Back to all news
arXiv Machine Learning By Egor Shvetsov, Aleksandr Serkov, Shokorov Viacheslav, Redko Dmitry, Vladislav Goloshchapov, Evgeny Burnaev

Bug or Feature$^2$: Weight Drift and Activation Sparsity

Read the original on arXiv Machine Learning →

The paper investigates a negative weight drift that arises from the interaction between standard loss functions (MSE or cross‑entropy) and positively biased activation functions. This drift, which is inherent to the optimization process rather than the data, pushes downstream weights toward negative values during early training and leads to high activation sparsity—up to nearly 90% in GPT‑nano when using ReLU. The authors also examine the impact of this sparsity, noting that model performance degrades sharply when activation sparsity exceeds about 70%, and study sparsity‑inducing activation functions that produce spikes in intermediate transformer layers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 9

Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes

Deep neural networks can exhibit periodic loss spikes known as the "Slingshot Mechanism" during long‑term training. This paper shows that such spikes arise from finite‑precision floating‑point arithmetic: when the correct‑class logit becomes much larger than the others, the gradient for that class is rounded to zero while others remain nonzero, breaking the zero‑sum constraint and creating a positive feedback loop that inflates feature and classifier norms. The authors term this Numerical Feature Inflation (NFI) and demonstrate that it explains the rapid norm growth, reappearance of gradients, and loss spikes, while also showing that partial absorption can still cause hidden norm growth without visible spikes.

By Hanqing Liu, Jianjun Cao, Yuanze Li, Zijian Zhou
More like this →
arXiv Machine Learning
6d ago

Flat Loss, Evolving Features: Grokking in Modular Addition

The paper demonstrates that neural networks can maintain perfect training accuracy while their training cross‑entropy remains nearly flat at a positive level, a phenomenon observed on modular addition tasks. Using ReLU networks with supported‑Polar hidden updates, normalized output‑layer updates, and weight decay, the authors show that hidden features evolve and held‑out accuracy can rise from zero to above 90% even as cross‑entropy stays flat. Theoretical results for one‑hidden‑layer networks and empirical evidence from deeper networks and transformers confirm that correct classification need not coincide with minimal cross‑entropy, challenging conventional assumptions about loss plateaus.

By Akash Kumar, Karthik Sankaranarayanan
llms
More like this →
arXiv Machine Learning
Oct 12

Simulating Training Spikes of Deep Neural Networks by Layer-wise Random Feature Models

The paper investigates intermittent spikes in loss observed during the pre‑training of large language models. It demonstrates that simple quadratic models with heavy‑tailed, power‑law distributed inputs can reproduce these spikes, and introduces a theoretical framework and simulation algorithm to model per‑layer training dynamics of deep neural networks. In a case study using an Adam configuration with β2 close to 1, the simulation predicts spiky behavior from a checkpoint and suggests a β2 schedule that empirically improves performance.

By Ran Tian, Courtney Paquette
llms
More like this →