arXiv Machine Learning July 7, 2026 By Yuri Kinoshita, Naoki Nishikawa, Taro Toyoizumi

Dataset Distillation Efficiently Encodes Low-Dimensional Representations from Gradient-Based Learning of Non-Linear Tasks

Read the original on arXiv Machine Learning →

arXiv:2603. 14830v3 Announce Type: replace Abstract: Dataset distillation, a training-aware data compression technique, has recently attracted increasing attention as an effective tool for mitigating costs of optimization and data storage.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

efficiency

arXiv Machine Learning

Jun 24

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination

arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.

By Peng Wang, Xiao Li, Can Yaras, Zhihui Zhu, Laura Balzano, Wei Hu, Qing Qu

fine-tuning

arXiv AI

Jul 7

Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) for Exponential Parameter Compression of Deep Neural Networks

arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.

By Andrzej Cichocki, Michal Wietczak

efficiency

arXiv Machine Learning

Jun 17

Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets?

arXiv:2606. 18209v1 Announce Type: new Abstract: Dataset distillation (DD) has emerged as a prominent approach in data centric machine learning, aiming to synthesize compact training sets for efficient training by compressing the information in large datasets into a small number of synthetic samples.

By Trisha Mittal, Akshay Mehra, Joshua Kimball

efficiency benchmarks

arXiv Machine Learning

Jul 16

Local Redundancy: An Information-Theoretic Measure of Plasticity from Synthetic Memorization

arXiv:2607. 13432v1 Announce Type: new Abstract: Plasticity -- a neural network's ability to adapt to new tasks -- is critical for continual and transfer learning.

By Jiaxuan Cheng

computer-vision fine-tuning

arXiv AI

Jul 9

Enhancing deep learning models for time series classification via knowledge distillation

arXiv:2607. 06796v1 Announce Type: cross Abstract: Deep learning has achieved remarkable success in various domains including time series analysis, computer vision and natural language processing.

By Javidan Abdullayev, Maxime Devanne, Jonathan Weber, Germain Forestier

llms computer-vision nlp efficiency benchmarks

arXiv AI

Jun 30

FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

arXiv:2603. 25144v2 Announce Type: replace-cross Abstract: Dataset distillation (DD) compresses a large training set into a small synthetic set, reducing storage and training cost, and has shown strong results on general benchmarks.

By Hongxu Ma, Guang Li, Shijie Wang, Dongzhan Zhou, Baoli Sun, Takahiro Ogawa, Miki Haseyama, Zhihui Wang

efficiency benchmarks

Related stories

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination

Automatically Differentiable Nonlinear Tensor Networks (ADNTNs) for Exponential Parameter Compression of Deep Neural Networks

Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets?

Local Redundancy: An Information-Theoretic Measure of Plasticity from Synthetic Memorization

Enhancing deep learning models for time series classification via knowledge distillation

FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation