arXiv AI
Jun 9
EinSort: Sorting is All We Need for Tensorizing LLM
arXiv:2606. 08565v1 Announce Type: cross Abstract: Tensor networks provide efficient representations for compressing large neural networks.
Tensor Data Scattering and the Impossibility of Slicing Theorem
arXiv:2012. 01982v3 Announce Type: replace Abstract: This paper proposes a standard way to represent sparse tensors.
Related stories
arXiv Machine Learning
Jul 9
VTC: DNN Compilation with Virtual Tensors for Data Movement Elimination
arXiv:2604. 09558v2 Announce Type: replace-cross Abstract: With the widening gap between compute and memory operation latencies, data movement optimizations have become increasingly important for DNN compilation.
arXiv AI
Aug 19
Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions
arXiv:2608. 17135v1 Announce Type: cross Abstract: Tensor networks are powerful formats for compressing large-scale data.
arXiv AI
Jun 3
Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
arXiv:2606. 03465v1 Announce Type: cross Abstract: Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints.
Hugging Face Trending Papers
Jun 2
Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures.
arXiv Machine Learning
Aug 31
A Deeper Analysis of Block-Sparse Featurizers
The paper investigates the block-sparse featurizer (BSF), a model that uses small subspaces as atomic units instead of single directions, aiming to capture features on low-dimensional manifolds common in vision. It identifies that BSF still exhibits classic sparse autoencoder failure modes such as feature splitting and composition. The authors propose architectural improvements, notably a Tournament Top‑K selection rule, which markedly reduces feature splitting, and they extend the block concept to a crosscoder framework.
arXiv Machine Learning
Aug 28
ReLATE: Accelerating Tensor Decomposition via Safe and Efficient Learning of Sparse Encodings
ReLATE is a reinforcement‑learned framework that automatically discovers safe and efficient sparse encodings for tensor decomposition, eliminating the need for expert‑designed formats. It combines model‑free and model‑based learning, elastic training, rule‑driven action masking, and dynamics‑informed filtering to guarantee correct encoding with bounded execution time, even early in training. After offline training, ReLATE deploys the optimal encoding with negligible overhead and achieves up to 2× speedups over the best expert‑designed format, with a geometric‑mean speedup of 1.38–1.41× on diverse real‑world sparse tensors.
arXiv Machine Learning
Jul 1
Online TT-ALS for Streaming Tensor Decomposition with Incremental Orthogonalization
arXiv:2606. 31061v1 Announce Type: cross Abstract: Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data.
arXiv Machine Learning
Jul 8
Separation Capacity of Scattering Networks on Low-Dimensional Datasets
arXiv:2607. 06048v1 Announce Type: cross Abstract: We aim to identify scattering network architectures that maximize the separation capacity on data with low intrinsic dimension.
Hugging Face Trending Papers
Aug 3
CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization
Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) performance in visual generative modeling, yet their training remains computationally prohibitive. While the recently proposed Momentum Orthogonalization (Muon) optimizer offers a promising alternative to AdamW, its direct application to DiTs yields suboptimal late-stage convergence.
arXiv Machine Learning
Aug 12
Accelerated Learning of High Dimensional Functions with a Tensor-Featured Training Network
arXiv:2608. 10351v1 Announce Type: new Abstract: In this work we present a method to accelerate the optimization of learning high dimensional functions using deep neural network (DNN).
arXiv AI
6d ago
DanLing NestedTensor: Composable Multi-Ragged Tensors for Deep Learning
DanLing NestedTensor is a PyTorch tensor abstraction that embeds multi‑ragged structure directly into the tensor, allowing packed values to carry partition information and logical dimension order. This design enables broadcasting, feature transformations, and reductions to automatically respect ragged axes while preserving the same representation through autograd and both eager and compiled execution. Benchmarks on an A100 show significant speedups—up to 3.39× over padding for BERT models and 2.40–4.32× for a Pairformer‑style workload—while dramatically reducing peak memory usage.