arXiv Machine Learning
Pre-Warm: Initializing Convolutional Filters from First-Batch Patch Dictionaries
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
Jun 24
Pre-Warm: Input-Conditioned Weight Initialization for Convolutional Neural Networks
We introduce Pre-Warm, a simple yet effective zero-training-cost method for data-conditioned initialization of the first convolutional layer. Before the first forward pass, Pre-Warm extracts mean-centered local patches from a single training batch, clusters them with MiniBatchKMeans, applies inverse Manhattan spatial weighting, and uses the resulting centroids to initialize half of the first-layer filters (the remainder retain Kaiming initialization).
arXiv Machine Learning
Jun 25
Pre-Warm: Input-Conditioned Weight Initialization for Convolutional Neural Networks
arXiv:2606. 25256v1 Announce Type: cross Abstract: We introduce Pre-Warm, a simple yet effective zero-training-cost method for data-conditioned initialization of the first convolutional layer.
arXiv Machine Learning
4d ago
Procedural Core: A Compact Recurrent Initialization for Vision Transformers
arXiv:2609.37631v1 Announce Type: new Abstract: Transformers are typically trained from random initialization, requiring all their capabilities to emerge from large-scale optimization. Recent work sh...
arXiv AI
Jul 8
Few-Medoids: An Embarrassingly Simple Coreset Selection Method for Few-Shot Knowledge Distillation
arXiv:2607. 05891v1 Announce Type: cross Abstract: Coreset selection aims to identify a small and highly representative subset of a massive dataset for efficient model training.
Hugging Face Trending Papers
Jul 22
Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training
Visible-infrared (VIS-IR) alignment is a key pre-training task for robust multi-sensor perception. Most existing methods use uniform patch-wise contrastive learning, but this can be unreliable in VIS-IR data because imaging-physics differences make some spatially paired regions inherently less comparable, and aligning them with equal strength hinders representation learning and downstream transfer.
Hugging Face Trending Papers
Aug 11
Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity
Vision Transformers underperform convolutional networks when training data is scarce, and distilling convolutional inductive biases from a CNN teacher is an effective remedy that leaves the deployed model unchanged. General-purpose feature distillation, however, transfers little in this setting.