arXiv:2603. 12222v2 Announce Type: replace-cross Abstract: Vision Transformers require significant computational resources and memory bandwidth, severely limiting their deployment on resource-constraint hardware.
arXiv:2606. 12278v1 Announce Type: cross Abstract: Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance.
Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance. Although the Lottery Ticket Hypothesis (LTH) shows that sparse subnetworks can match dense networks when trained from suitable initializations, its iterative pruning procedure requires multiple complete training cycles.
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv:2608. 08624v1 Announce Type: new Abstract: Domain generalization (DG) and neural network pruning are conventionally treated as distinct objectives, targeting out-of-distribution (OOD) robustness and model efficiency, respectively.
LearnPrunerV2 is a fully learnable token‑pruning framework for Vision‑Language Models that replaces heuristic pruning with lightweight modules in every backbone layer. These modules estimate token importance by combining visual saliency and query relevance via a shadow token, revealing a consistent staircase redundancy pattern across layers. The method introduces Budget‑Deterministic Pruning to set fixed per‑layer budgets, achieving state‑of‑the‑art efficiency while preserving 99.8% of performance even after pruning 77.8% of visual tokens on LLaVA‑v1.5.
arXiv:2411. 16073v4 Announce Type: replace-cross Abstract: Inspired by the Well-initialized Lottery Ticket Hypothesis (WLTH), we introduce Soft-TransFormers (Soft-TF), a continual learning framework that adapts a frozen pre-trained Transformer through task-specific soft subnetworks: real-valued multiplicative masks over the query, key, value, and output projections of selected self-attention layers.
arXiv:2607. 15745v1 Announce Type: new Abstract: Common practice when training Convolutional Neural Networks (CNNs) is to use randomly shuffled mini-batches.
arXiv:2506. 14126v2 Announce Type: replace-cross Abstract: Modern deep learning is increasingly characterized by the use of open-weight foundation models that can be fine-tuned on specialized datasets.
arXiv:2608.28923v1 Announce Type: cross Abstract: Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, eithe...
The paper introduces a new active learning signal for object detection that relies on a supervised contrastive term added to the training objective. This term shapes an embedding space where distance reflects class membership, allowing an unlabeled detection to be scored by its distance from the predicted category’s region weighted by confidence—all from a single forward pass of one network. Experiments on PASCAL VOC and MS‑COCO show that this criterion outperforms the standard posterior and remains competitive with ensemble‑based methods while incurring only a modest 8.3% increase in parameters.
The paper introduces ActiveKD, a framework that combines active learning with knowledge distillation by using large vision‑language models as teachers. It leverages the structured prediction bias of these models to design Probabilistic CoreSet (PCoreSet), a selection strategy that maximizes coverage in probability space rather than feature space. Experiments on 11 datasets show that ActiveKD improves all selection methods over a no‑distillation baseline, and PCoreSet consistently outperforms random sampling across multiple students and teachers.