arXiv Machine Learning

Layer-wise Derivative Controlled Networks Achieve Competitive Accuracy and Gradient Stability Across Data Regimes

arXiv:2606. 07908v1 Announce Type: new Abstract: Derivative-controlled networks based on ChainzRule (CR) combine cubic polynomial layers with a lightweight forward-mode per-layer Jacobian penalty (DREG).

arXiv Machine Learning

Jun 24

DREG: A Layer-Wise Jacobian Regularization as a General-Purpose Penalty

arXiv:2606. 23942v1 Announce Type: new Abstract: We present a large-scale empirical study isolating the contributions of the Derivative Regularization penalty (DREG).

By Rowan Martnishn

llms nlp benchmarks safety

arXiv Machine Learning

Aug 11

Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

arXiv:2608. 09523v1 Announce Type: new Abstract: Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success.

By Binchuan Qi

safety

arXiv Machine Learning

Jun 30

Improved Predictive Performance and Interpretability for Mesomorphic Neural Networks Using Local Fidelity Regularization

arXiv:2606. 29951v1 Announce Type: new Abstract: Interpretable Mesomorphic Neural Networks (IMNs) offer a promising framework that combines the predictive power of deep neural networks with the interpretability of linear models.

By Hugo L. Hammer, Vajira Thambawita, Kristoffer Herland Hellton, P{\aa}l Halvorsen

benchmarks safety

arXiv Machine Learning

Jun 29

Layerwise Progressive Freezing: A Training Scaffold for Depth-Scalable Binary Networks

arXiv:2606. 27759v1 Announce Type: new Abstract: Training binary neural networks (BNNs) from scratch is dominated by the straight-through estimator (STE), whose forward/backward mismatch produces severe accuracy degradation as networks deepen.

By Evan Gibson Smith, Bashima Islam

fine-tuning

arXiv Machine Learning

Jun 10

Overcoming Rank Collapse in Feedback Alignment

arXiv:2606. 11123v1 Announce Type: new Abstract: Backpropagation (BP) is widely viewed as biologically implausible, in part because it requires feedback weights to be the transpose of forward weights for error propagation.

By Gauthier Boeshertz, Razvan Pascanu, Claudia Clopath

benchmarks safety

arXiv Machine Learning

Jul 10

LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks

arXiv:2607. 07745v1 Announce Type: new Abstract: While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge.

By Arthur Chiron (IRIT, EPE UT), Franck Mamalet (IRIT, DTIPG - SNCF, UT3), Thomas Massena (IRIT, DTIPG - SNCF, UT3), Thomas Deltort (IRIT), Mathieu Serrurier (IRIT, UT2J)

benchmarks

arXiv Machine Learning

Jul 1

A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning

arXiv:2602. 20062v2 Announce Type: replace Abstract: Pretraining and fine-tuning are central stages in modern machine learning systems.

By Nicolas Anguita, Francesco Locatello, Andrew M. Saxe, Marco Mondelli, Flavia Mancini, Samuel Lippl, Clementine Domine

llms fine-tuning safety

arXiv Machine Learning

Jul 7

Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models

arXiv:2607. 04775v1 Announce Type: cross Abstract: Score-based Generative Models (SGMs) have achieved impressive performance in data generation across a wide range of applications.

By Stanislas Strasman (SU, LPSM), Sobihan Surendran (SU, LPSM), Sylvain Le Corff (SU, LPSM)

diffusion

arXiv Machine Learning

Jul 2

ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces

arXiv:2607. 01125v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization enables fine-tuning large language models when backpropagation is unavailable or memory-prohibitive, but existing methods often perturb full model weights or randomly constructed low-dimensional subspaces, yielding high-variance estimates and limited performance.

By Xun Dong, Yibo Xu, Naigang Wang, Xin Li, Penghang Yin, Zi Yang

llms nlp fine-tuning efficiency safety

arXiv Machine Learning

Jun 24

XConv: Low-memory stochastic backpropagation for convolutional layers

arXiv:2106. 06998v5 Announce Type: replace Abstract: Training convolutional neural networks at scale demands substantial memory, largely because intermediate activations must be stored for backpropagation.

By Anirudh Thatipelli, Jeffrey Sam, Mathias Louboutin, Ali Siahkoohi, Rongrong Wang, Felix J. Herrmann

computer-vision fine-tuning

arXiv AI

Jul 29

CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning

arXiv:2509. 11285v2 Announce Type: replace-cross Abstract: Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting.

By Alejandro Dopico-Castro, Oscar Fontenla-Romero, Bertha Guijarro-Berdi\~nas, Amparo Alonso-Betanzos

rag diffusion safety

arXiv Machine Learning

Jun 5

Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction

arXiv:2606. 05863v1 Announce Type: new Abstract: Grokking suggests that fitting the training data and learning a simple underlying rule may occur on different time scales.

By Hu Tan, Kuo Gai, Shihua Zhang

rag

Related stories