arXiv Machine Learning
Jul 9
LEMUR 2: Unlocking Neural Network Diversity for AI
arXiv:2607. 06839v1 Announce Type: new Abstract: Existing NAS benchmarks (e.
Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesis
Introduce a MinHash-based similarity scheduling framework that constructs a progressive curriculum over neural architecture code for LLM-based neural architecture search (NAS). Using 128-permutation MinHash signatures over normalised 7-gram source code shingles, we partition the reference pool into similarity bands and present them in increasing architectural heterogeneity, with the best LoRA adapter from each stage merged cumulatively into the backbone.
Related stories
arXiv AI
Jul 20
When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models
arXiv:2603. 26556v2 Announce Type: replace-cross Abstract: Converting a pretrained Transformer into a more efficient hybrid model through distillation offers a promising approach to reducing inference costs.
arXiv AI
Jun 18
From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging
arXiv:2506. 14126v2 Announce Type: replace-cross Abstract: Modern deep learning is increasingly characterized by the use of open-weight foundation models that can be fine-tuned on specialized datasets.
Hugging Face Trending Papers
Jul 9
Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures
Choosing a learning rate scheduling strategy is critical to neural network training, but manual selection is costly and rarely exhaustive. While classical AutoML approaches often treat the scheduler as a secondary hyperparameter, we systematically investigate its impact on classification accuracy across a diverse pool of architectures.
arXiv Machine Learning
Jun 16
Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
arXiv:2601. 18699v2 Announce Type: replace Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.
arXiv Machine Learning
Jul 27
Layer-wise LoRA fine-tuning: a similarity metric approach
arXiv:2602. 05988v2 Announce Type: replace Abstract: Pre-training Large Language Models (LLMs) on web-scale datasets becomes fundamental for advancing general-purpose AI.
arXiv AI
Jun 9
Seq103: A Unified Neuroevolution Framework for Compact Sequence Architecture Discovery
arXiv:2606. 07664v1 Announce Type: cross Abstract: Neuroevolution is a representative neural architecture search paradigm that evolves both network topology and weights through evolutionary algorithms.
arXiv AI
Jun 29
The Context-Ready Transformer
arXiv:2606. 27538v1 Announce Type: cross Abstract: We introduce the context-ready transformer, a new recurrent neural network architecture built from a D-layer transformer block that pre-contextualizes each token before it enters the block.
arXiv AI
5d ago
Mixture of Training: Recombining Small-Scale Scaffolded Pretraining Runs into a Larger Language Model
arXiv:2608. 13277v1 Announce Type: cross Abstract: We ask whether language-model pre-training can be decomposed into smaller, independently trainable jobs that can later be recomposed into a coherent larger model.
arXiv Machine Learning
Jul 10
Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures
arXiv:2607. 08511v1 Announce Type: new Abstract: Choosing a learning rate scheduling strategy is critical to neural network training, but manual selection is costly and rarely exhaustive.
arXiv Machine Learning
Jun 5
GenFT: A Generative Parameter-Efficient Fine-Tuning Method for Pretrained Foundation Models
arXiv:2506. 11042v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a resource-efficient strategy for adapting Pretrained Foundation Models (PFMs) by learning a small number of task-specific updates $\Delta W$.
arXiv AI
Aug 11
UniDFKD: A Unified Semantic Prior Framework for Architecture-Agnostic Data-Free Knowledge Distillation
arXiv:2608. 09287v1 Announce Type: cross Abstract: Data-Free Knowledge Distillation (DFKD) transfers knowledge from a pretrained teacher model to a compact student model by synthesizing semantically informative data, eliminating the need for access to the original training dataset.