arXiv Machine Learning

Random Label Prediction Heads for Studying Memorization in Deep Neural Networks

arXiv:2607. 11541v1 Announce Type: new Abstract: We introduce a straightforward yet effective method to empirically study memorization in deep neural networks for classification tasks.

arXiv Machine Learning
Jul 30

The Advantage of Fine-Grained Training

arXiv:2509. 05130v2 Announce Type: replace Abstract: In classification problems, models are trained to predict a class label based on the input data features.

By Davide Pirovano, Federico Milanesio, Michele Caselle, Piero Fariselli, Matteo Osella
arXiv Machine Learning
Jul 7

Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training

arXiv:2607. 04969v1 Announce Type: new Abstract: The training paradigm of large language models has shifted from traditional one-pass training to multi-epoch training, as reasonable reuse of limited high-quality data can improve both model performance and sample efficiency.

By Jingwei Zuo, Cong Zeng, Ilyas Chahed, Maksim Velikanov, Dhia Eddine Rhaiem, Pasquale Balsebre, Abhay Kumar, Younes Belkada, Hakim Hacid
arXiv AI
Jun 29

Applicability of memorization indicators for early spotting of overfitting while recalibrating sEMG-decoders on low sample sizes

arXiv:2606. 27855v1 Announce Type: cross Abstract: Deep learning models for surface electromyography (sEMG) can benefit substantially from subject-specific (re-)calibration, since no sufficiently large and diverse datasets are available to train fully generic decoders.

By Stephan J. Lehmler, Tobias Glasmachers, Ioannis Iossifidis