arXiv:2602. 02759v3 Announce Type: replace-cross Abstract: Despite the ubiquity of multiway data across scientific domains, there are few performant and user-friendly methods that fit non-standard nonnegative tensor factorization models tailored to the data at-hand.
arXiv:2607. 13919v1 Announce Type: new Abstract: Nonnegative Matrix Factorization (NMF) is a fundamental tool in unsupervised learning, which approximates a nonnegative matrix by the product of two low-rank nonnegative factors.
arXiv:2607. 27987v1 Announce Type: new Abstract: Tensor operations represent a cornerstone of modern scientific computing.
This survey reviews tensor methods applied to large language models, framing them through a seven‑stage lifecycle (tokenization, embeddings, pre‑training, adaptation, compression, inference, interpretability) and a component view (embeddings, attention, feed‑forward networks). It offers unified notation, theoretical foundations, and comparative analyses of tensorization strategies for Transformer components, while highlighting evaluation protocol differences and model scale effects. The paper also introduces a new metric, ρ_gap, to quantify the gap between theoretical memory savings and actual system‑level speedup, and connects tensor techniques to related efficiency and probabilistic methods.
arXiv:2405. 18220v4 Announce Type: replace-cross Abstract: Tensor-based discrete density estimation requires flexible modeling and proper divergence criteria to enable effective learning; however, traditional approaches using $\alpha$-divergence face analytical challenges due to the $\alpha$-power terms in the objective function, which hinder the derivation of closed-form update rules.
arXiv:2606. 25975v1 Announce Type: new Abstract: Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models.
The paper introduces Coupled Tensor‑Tensor Completion (CTTC), a new framework that incorporates side information in tensor form to enhance tensor completion tasks. CTTC leverages hidden connections among multimodal tensors and is grounded in distance metric learning and group theory. Experiments on the DTD and LINCS datasets show that CTTC outperforms existing methods such as HaLRTC, CTRC, Cell, and NTDDR in both run‑time and root‑sum‑of‑errors accuracy for predicting drug effects.
Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is unde...
Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models. Recent work has shown that exploiting matrix structure can improve optimization dynamics.
The paper introduces Coupled Tensor‑Tensor Completion (CTTC), a new framework that incorporates side information in tensor form to enhance tensor completion tasks. CTTC leverages hidden connections among multimodal tensors and is grounded in distance metric learning and group theory. Experiments on the DTD and LINCS datasets show that CTTC outperforms existing methods such as HaLRTC, CTRC, Cell, and NTDDR in both runtime and root‑sum‑of‑squares error for drug effect prediction.
arXiv:2606. 31061v1 Announce Type: cross Abstract: Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data.
arXiv:2606. 00130v1 Announce Type: cross Abstract: We study Automatically Differentiable Nonlinear Tensor Networks (ADNTNs), a family of structured weight generators whose compact core tensors are trained end-to-end by reverse-mode automatic differentiation (AD).
arXiv:2104. 08928v4 Announce Type: replace-cross Abstract: Unstructured text provides decision-makers with a rich data source in many domains, ranging from product reviews in retail to nursing notes in healthcare.