arXiv AI June 2, 2026 By Andrzej Cichocki, Michal Wietczak

Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) for Exponential Compression of Deep Neural Networks

arXiv:2606. 00130v1 Announce Type: cross Abstract: We study Automatically Differentiable Nonlinear Tensor Networks (ADNTNs), a family of structured weight generators whose compact core tensors are trained end-to-end by reverse-mode automatic differentiation (AD).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 7

Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) for Exponential Parameter Compression of Deep Neural Networks

arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.

By Andrzej Cichocki, Michal Wietczak

efficiency

arXiv AI

22h ago

Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions

arXiv:2608. 17135v1 Announce Type: cross Abstract: Tensor networks are powerful formats for compressing large-scale data.

By Xiao Wang, Tomohiro Hashizume, Pia Siegl, Dieter Jaksch

arXiv Machine Learning

Jun 25

Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

arXiv:2606. 25971v1 Announce Type: new Abstract: Modern neural network training relies on optimizers such as Adam and Muon which act on each weight matrix as a single object.

By Alexander H\"agele, Alejandro Hern\'andez-Cano, Atli Kosson, Martin Jaggi

arXiv Machine Learning

Aug 4

Nova: An End-to-End MLIR Compiler for Deep Learning

arXiv:2608. 00029v1 Announce Type: cross Abstract: The performance of deep learning models at scale relies heavily on how effectively high-level mathematical operations are mapped to underlying physical hardware.

By Adwaid Suresh, Aparna A, Harshini V M, Jona Delcy C A, Killi Uma Maheswara Rao, Ram Charan Golla, Surendra Vendra

efficiency

arXiv AI

Jul 14

Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization

arXiv:2607. 09967v1 Announce Type: cross Abstract: Many neural networks operations have a multiplicative nature rather than additive: halving or doubling a norm are analogous relatively but require unequal optimization distances when taking linear steps.

By Ethan Smith

llms

arXiv Machine Learning

Jun 2

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.

By Cem \"Uy\"uk, Mike Lasby, Mohamed Yassin, Utku Evci, Yani Ioannou

llms fine-tuning efficiency benchmarks

Related stories

Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) for Exponential Parameter Compression of Deep Neural Networks

Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions

Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

Nova: An End-to-End MLIR Compiler for Deep Learning

Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition