arXiv AI August 14, 2026 By Andrew Cheng, Ali Eslamian, Jie Cheng, Mehdi Zargham, Qiang Cheng

Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks

arXiv:2608. 12597v1 Announce Type: cross Abstract: Neural networks can often be trained or fine-tuned through random low-dimensional reparameterization, where a small latent vector is mapped into a full parameter update by a frozen random map.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 4

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.

By Berk Tinaz, Changzhi Xie, Mahdi Soltanolkotabi

safety

arXiv Machine Learning

Jun 30

Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization

arXiv:2606. 30226v1 Announce Type: new Abstract: Hessian spectral properties are a standard tool in analysing neural-network training, with eigenvalues linked to sharpness, generalization, and optimization dynamics.

By Marcelina Marjankowska, Valerio Modugno, Paolo Barucca

arXiv AI

Jul 14

Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization

arXiv:2607. 09967v1 Announce Type: cross Abstract: Many neural networks operations have a multiplicative nature rather than additive: halving or doubling a norm are analogous relatively but require unequal optimization distances when taking linear steps.

By Ethan Smith

llms

arXiv AI

Jun 10

Recoverable but Not Stationary:Local Linear Structures in Weights and Activations

arXiv:2606. 10929v1 Announce Type: cross Abstract: Task vectors, LoRA, activation steering, and random search around pretrained weights all suggest that learned behaviour can be controlled by linear directions.

By Irina Piontkovskaia, Sergey Nikolenko

llms fine-tuning

arXiv Statistics ML

Sep 10

DFNN: A Deep Fr\'echet Neural Network Framework for Learning Metric-Space-Valued Responses

arXiv:2510.17072v2 Announce Type: replace Abstract: Regression with non-Euclidean responses---e.g., probability distributions, networks, symmetric positive-definite matrices, and compositions---has b...

By Kyum Kim, Yaqing Chen, Paromita Dubey

benchmarks

arXiv Machine Learning

4d ago

Traversing the solution space of neural networks with Hessian Null Space Continuation

arXiv:2609.38081v1 Announce Type: new Abstract: On a single task, deep networks can learn many solutions, depending on their optimizer, training data, architecture, and hyperparameters. Many of these...

By Ann Huang, Mitchell Ostrow, Zhouyang Lu, William T. Redman, Leo Kozachkov, Kanaka Rajan

llms agents fine-tuning safety

One email a morning, machine-written

Related stories

When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

Characterizing Optimizer-Dependent Training Dynamics Through Hessian Eigenvector Displacement and Localization

Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization

Recoverable but Not Stationary:Local Linear Structures in Weights and Activations

DFNN: A Deep Fr\'echet Neural Network Framework for Learning Metric-Space-Valued Responses

Traversing the solution space of neural networks with Hessian Null Space Continuation