arXiv Machine Learning

Semi-Tensor Product-Based Multi-Term Randomized T-SVD and Its Visual Applications

The paper introduces a new semi‑tensor product for third‑order tensors that relaxes the dimensional constraints of the standard t‑product while preserving the closed‑form nature of T‑SVD. It builds a multi‑term semi‑tensor product singular value decomposition (MSTP‑SVD) that improves low‑rank approximation accuracy, and further accelerates it with randomized projection and power iteration to create the MRSTP‑SVD algorithm. Experiments on image and video compression and completion show that this method balances reconstruction accuracy and computational efficiency.

arXiv Machine Learning
5d ago

TUON: A Tensor-Aware Optimizer Utilizing Tensor-SVD and Polarization for Large Language Model Pretraining

The paper introduces TUON, a tensor‑aware optimizer that leverages Tensor‑SVD and a unitary transform to mix and polarize layers, thereby capturing cross‑layer correlations while preserving per‑layer diversity. Unlike previous matrix‑aware optimizers such as Muon, TUON avoids structural imbalance and misalignment caused by unfolding, enabling scaling to more layers and diverse modules beyond QKV. The authors prove that TUON performs steepest descent under a transformed tensor spectral norm, establish theoretical properties relative to Muon and TEON, and demonstrate empirical improvements on GPT‑124M/350M and Qwen‑0.6B/4B models.

By Anda Tang, Xuanlin Yang, Junjie Wang, Zhouchen Lin
arXiv AI
6d ago

Shared-Factor-Conditioned Diffusion for Low-Rank Tensor Recovery

The paper introduces DiffTR, an unsupervised framework that unifies tensor completion, denoising, and hyperspectral super‑resolution using coordinate‑conditioned diffusion. Shared factors indexed by tensor coordinates condition a compact score network, allowing the model to learn both the factors and an implicit tensor representation from observations. The framework separates representation learning from task‑specific observation modeling, enabling reverse diffusion for missing‑value recovery, Tweedie‑based denoising, and measurement‑guided sampling for high‑resolution reconstruction, all while supporting multiple tensor modalities and corruption models.

By Zhengyun Cheng, Guanwen Zhang, Wei Zhou
Hugging Face Trending Papers
Jul 27

Structural Loss Metrics for Tensor Approximation via Matrix Low-Rank Approximation

Matricized low-rank approximation via SVD is a standard surrogate for tensor decompositions, but entry-wise reconstruction error fails to capture multiway geometric degradation. Under an orthogonal Tucker model, we characterize this degradation using two metrics: cross-mode Direction Loss, measuring geometric subspace deviation from rank truncation and noise rotation, and Interaction Loss, quantifying multilinear interaction distortion in the core tensor.

arXiv Machine Learning
Jun 25

Low-Cost High-Order Singular Value Decomposition for Tensor-Based Reconstruction from Sparse Sensor Measurements: Urban Flow and Air-Quality Applications

arXiv:2606. 24989v1 Announce Type: new Abstract: Urban flow and air-quality simulations generate high-dimensional datasets describing velocity and pollutant transport across multiple spatial, temporal, and physical-variable dimensions.

By Arindam Sengupta, Paul Jeanney, Ricardo Vinuesa, Jose Miguel Perez, Soledad Le Clainche
Hugging Face Trending Papers
Sep 2

Coupled Tensor-Tensor Completion Method with Applications in Drug Repurposing

The paper introduces Coupled Tensor‑Tensor Completion (CTTC), a new framework that incorporates side information in tensor form to enhance tensor completion tasks. CTTC leverages hidden connections among multimodal tensors and is grounded in distance metric learning and group theory. Experiments on the DTD and LINCS datasets show that CTTC outperforms existing methods such as HaLRTC, CTRC, Cell, and NTDDR in both run‑time and root‑sum‑of‑errors accuracy for predicting drug effects.