arXiv Machine Learning

Tensor Network Kernel Machines: A JAX Framework for Machine Learning and Nonlinear System Identification

arXiv:2608. 07043v1 Announce Type: cross Abstract: Developing nonlinear models that are both expressive and computationally efficient remains a challenge in machine learning and nonlinear system identification.

arXiv Machine Learning
Sep 23

Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning

The paper presents a formal analysis of the quotient geometry of tree tensor networks (TTNs) and introduces efficient first- and second-order optimization algorithms that leverage this geometry. It also develops a backpropagation method for training TTNs in a kernel learning context. Numerical experiments on a digit classification task demonstrate a tradeoff between two horizontal distributions: one provides clearer geometric insights, while the other yields more efficient algorithms.

By Marius Willner, Marco Trenti, Dirk Lebiedz
arXiv AI
Sep 15

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.

By Safa Hamreras, Sukhbinder Singh, Rom\'an Or\'us
arXiv AI
Sep 16

Continuous-Time Machine Learning: A Unified Mathematical Perspective

The paper surveys continuous‑time (CT) machine learning, a framework for modeling temporal dynamics as continuous processes, especially useful when data are sampled irregularly or over long horizons. It introduces a unified taxonomy that groups major CT methods by their underlying mathematical formulations and shows how different architectural choices—such as vector‑field parameterization, stochasticity, memory mechanisms, and discretization—relate these families. The survey compares training algorithms, optimization strategies, failure modes, computational complexity, and benchmarks, reviews supporting software ecosystems, and outlines open challenges and future research directions.

By Waleed Razzaq, Yun-Sheng Zhao, Yun-Bo Zhao