The paper introduces DiffTR, an unsupervised framework that unifies tensor completion, denoising, and hyperspectral super‑resolution using coordinate‑conditioned diffusion. Shared factors indexed by tensor coordinates condition a compact score network, allowing the model to learn both the factors and an implicit tensor representation from observations. The framework separates representation learning from task‑specific observation modeling, enabling reverse diffusion for missing‑value recovery, Tweedie‑based denoising, and measurement‑guided sampling for high‑resolution reconstruction, all while supporting multiple tensor modalities and corruption models.
By Zhengyun Cheng, Guanwen Zhang, Wei Zhou
The paper introduces a new semi‑tensor product for third‑order tensors that relaxes the dimensional constraints of the standard t‑product while preserving the closed‑form nature of T‑SVD. It builds a multi‑term semi‑tensor product singular value decomposition (MSTP‑SVD) that improves low‑rank approximation accuracy, and further accelerates it with randomized projection and power iteration to create the MRSTP‑SVD algorithm. Experiments on image and video compression and completion show that this method balances reconstruction accuracy and computational efficiency.
By Xingchen Xiao (School of Mathematics and Statistics, Southwest University, Chongqing, China), Feng Zhang (School of Mathematics and Statistics, Southwest University, Chongqing, China), Wenjin Qin (School of Mathematics and Statistics, Southwest University, Chongqing, China), Jianjun Wang (School of Mathematics and Statistics, Southwest University, Chongqing, China)
arXiv:2606. 25975v1 Announce Type: new Abstract: Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models.
By Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko, Sergei Kudriashov, Maxim Rakhuba
Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models. Recent work has shown that exploiting matrix structure can improve optimization dynamics.
The paper proposes that super‑resolution should be performed in a latent space that is close to the manifold of high‑resolution images, rather than in reconstruction‑oriented or pixel spaces. By using the latent space of 23 fused layers of a frozen DINOv3‑L model, degraded images are embedded near this manifold, enabling a 415M decoder (RAESR) to map them back and decode to pixels in a single pass. RAESR achieves the best fidelity–perception trade‑off on several benchmarks while running at 37 ms per 512×512 image on a single H20 GPU, outperforming models that use a VAE latent substrate.
By Wanzhou Lei, Cuifeng Shen, Yanjin He, Maohua Li, Hua Yuan, Per-Olof Persson, Tao Lan, Kan Liu, Hanlin Tang
This survey reviews tensor methods applied to large language models, framing them through a seven‑stage lifecycle (tokenization, embeddings, pre‑training, adaptation, compression, inference, interpretability) and a component view (embeddings, attention, feed‑forward networks). It offers unified notation, theoretical foundations, and comparative analyses of tensorization strategies for Transformer components, while highlighting evaluation protocol differences and model scale effects. The paper also introduces a new metric, ρ_gap, to quantify the gap between theoretical memory savings and actual system‑level speedup, and connects tensor techniques to related efficiency and probabilistic methods.
By Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida, Andrzej Cichocki
The paper introduces TUON, a tensor‑aware optimizer that leverages Tensor‑SVD and a unitary transform to mix and polarize layers, thereby capturing cross‑layer correlations while preserving per‑layer diversity. Unlike previous matrix‑aware optimizers such as Muon, TUON avoids structural imbalance and misalignment caused by unfolding, enabling scaling to more layers and diverse modules beyond QKV. The authors prove that TUON performs steepest descent under a transformed tensor spectral norm, establish theoretical properties relative to Muon and TEON, and demonstrate empirical improvements on GPT‑124M/350M and Qwen‑0.6B/4B models.
By Anda Tang, Xuanlin Yang, Junjie Wang, Zhouchen Lin
arXiv:2606. 03212v1 Announce Type: new Abstract: Low-rank tensor decomposition (TD) is usually effective on clean, fully observed data, but it often degrades under severe missingness or noise.
By Zerui Tao, Qibin Zhao
Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is unde...
arXiv:2608. 10351v1 Announce Type: new Abstract: In this work we present a method to accelerate the optimization of learning high dimensional functions using deep neural network (DNN).
By Karl Pierce, Yuehaw Khoo, Haizhao Yang
arXiv:2606. 31061v1 Announce Type: cross Abstract: Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data.
By Hiroki Takeda, Yuto Miyatake, Daisuke Furihata
arXiv:2606. 08565v1 Announce Type: cross Abstract: Tensor networks provide efficient representations for compressing large neural networks.
By Toshiaki Koike-Akino, Jing Liu, Ye Wang