The paper introduces a new semi‑tensor product for third‑order tensors that relaxes the dimensional constraints of the standard t‑product while preserving the closed‑form nature of T‑SVD. It builds a multi‑term semi‑tensor product singular value decomposition (MSTP‑SVD) that improves low‑rank approximation accuracy, and further accelerates it with randomized projection and power iteration to create the MRSTP‑SVD algorithm. Experiments on image and video compression and completion show that this method balances reconstruction accuracy and computational efficiency.
By Xingchen Xiao (School of Mathematics and Statistics, Southwest University, Chongqing, China), Feng Zhang (School of Mathematics and Statistics, Southwest University, Chongqing, China), Wenjin Qin (School of Mathematics and Statistics, Southwest University, Chongqing, China), Jianjun Wang (School of Mathematics and Statistics, Southwest University, Chongqing, China)
arXiv:2606. 10085v1 Announce Type: new Abstract: Matrix-valued time series arise in a wide range of applications, such as spatio-temporal data from medical imaging and geophysics.
By Zhen Qin, Yang Chen
arXiv:2608. 17135v1 Announce Type: cross Abstract: Tensor networks are powerful formats for compressing large-scale data.
By Xiao Wang, Tomohiro Hashizume, Pia Siegl, Dieter Jaksch
Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models. Recent work has shown that exploiting matrix structure can improve optimization dynamics.
arXiv:2607. 25967v2 Announce Type: replace-cross Abstract: Singular Value Decomposition (SVD) underlies matrix factorisation tasks across many fields, with imaging applications demanding real-time processing.
By Christopher Hahne
arXiv:2606. 25975v1 Announce Type: new Abstract: Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models.
By Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko, Sergei Kudriashov, Maxim Rakhuba
arXiv:2609.12843v1 Announce Type: new
Abstract: Recently, tensor decompositions are prevalent for multi-dimensional image representation, which learn the instance-specific structure of each image fro...
By Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang, Xi-Le Zhao, Deyu Meng
arXiv:2412. 07041v4 Announce Type: replace-cross Abstract: Recovering incomplete multidimensional tensor-structured data is a fundamental task in many real-world applications.
By Mengying Lei, Lijun Sun
arXiv:2608. 03928v1 Announce Type: cross Abstract: Tensor cross-concentrated sampling (t-CCS) bridges entrywise sampling and t-CUR slice-wise sampling by observing entries only within selected horizontal and lateral slices.
By Hanqin Cai, Longxiu Huang, Jing Qin, Chengyue Wu
arXiv:2608. 10351v1 Announce Type: new Abstract: In this work we present a method to accelerate the optimization of learning high dimensional functions using deep neural network (DNN).
By Karl Pierce, Yuehaw Khoo, Haizhao Yang
Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) performance in visual generative modeling, yet their training remains computationally prohibitive. While the recently proposed Momentum Orthogonalization (Muon) optimizer offers a promising alternative to AdamW, its direct application to DiTs yields suboptimal late-stage convergence.
The paper introduces FACTS, a structured Fisher Approximation for compressing Vision Transformers (ViTs) using Fisher-weighted SVD, which enforces token‑local aggregation while preserving within‑token activation‑gradient dependence. It also presents Constrained Rank Search (CoRS) to optimize layer‑wise rank allocation under a fixed FLOP budget. Experiments on ViTs and hybrid architectures show that FACTS improves accuracy‑efficiency trade‑offs, outperforming the strongest SVD baseline by up to +5.8 percentage points on Swin‑B without requiring finetuning.
By Moritz Thoma, Maximilian Groezinger, Maximilian Forstenh\"ausler, Emad Aghajanzadeh, Ryan Pegoud, Manoj Rohit Vemparala, Pierpaolo Mori, Alexander Frickenstein, Daniel Mueller-Gritschneder, Ulf Schlichtmann