← Back to all news
arXiv AI

FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

arXiv:2603. 25144v2 Announce Type: replace-cross Abstract: Dataset distillation (DD) compresses a large training set into a small synthetic set, reducing storage and training cost, and has shown strong results on general benchmarks.

Related stories

arXiv Machine Learning
Jul 30

The Advantage of Fine-Grained Training

arXiv:2509. 05130v2 Announce Type: replace Abstract: In classification problems, models are trained to predict a class label based on the input data features.

By Davide Pirovano, Federico Milanesio, Michele Caselle, Piero Fariselli, Matteo Osella
safety
More like this →
arXiv Computer Vision
Aug 27

DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation

DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation introduces a method that compresses large training sets into compact synthetic sets while preserving fine-grained visual classification cues. It uses attention rollout from a pretrained TransFG teacher to locate informative patches, applies spatial diversification to avoid redundancy, and organizes these patches into class-wise evidence banks that are packed into grid-composed images. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars demonstrate that DeCO outperforms existing coreset and dataset-distillation baselines across various images-per-class budgets.

By Chuixuan Fan, Guang Li, Shijie Wang, Dongzhan Zhou, Baoli Sun, Takahiro Ogawa, Miki Haseyama, Zhihui Wang
efficiency
More like this →
arXiv Machine Learning
Jun 17

Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets?

arXiv:2606. 18209v1 Announce Type: new Abstract: Dataset distillation (DD) has emerged as a prominent approach in data centric machine learning, aiming to synthesize compact training sets for efficient training by compressing the information in large datasets into a small number of synthetic samples.

By Trisha Mittal, Akshay Mehra, Joshua Kimball
efficiencybenchmarks
More like this →
arXiv AI
Aug 26

Too much of a good thing -- when knowledge distillation promotes overfitting, and how to avoid it

The paper investigates how knowledge distillation (KD) applied at intermediate layers of a neural network can affect overfitting and model performance. While traditional KD focuses on the final output, this study explores block‑wise KD across eleven datasets, finding that on standard datasets the last block suffices, but on fine‑grained, data‑scarce settings intermediate supervision significantly improves accuracy. The authors also analyze optimal supervision granularity using attention maps, Centered Kernel Alignment, and Grad‑CAM, and examine teacher‑student fine‑tuning strategies.

By Irene Trigueros-Lorca, Leonardo Concepci\'on, Christian Wagner, Isaac Triguero, Daniel Molina
fine-tuningefficiencysafety
More like this →
arXiv Machine Learning
Sep 23

eXplaining to Learn (eX2L): Regularization Using Contrastive Visual Explanation Pairs for Distribution Shifts

The paper introduces eXplaining to Learn (eX2L), an interpretable framework that regularizes a classifier by penalizing similarity between Grad‑CAM maps of the main label classifier and a confounder classifier. This approach decorrelates confounding features from latent representations during training. On the Spawrious Many‑to‑Many Hard Challenge benchmark, eX2L outperforms the current state‑of‑the‑art by 5.49% in average accuracy and 10.90% in worst‑group accuracy, while also demonstrating functional domain invariance through explicit label‑nuisance decoupling.

By Paulo Mario P. Medina, Jose Marie Antonio Mi\~noza, Sebastian C. Iba\~nez
benchmarkssafety
More like this →