arXiv:2503. 07325v2 Announce Type: replace Abstract: Understanding and certifying the behavior of modern deep neural networks remains a fundamental challenge in reliable machine learning.
arXiv:2603. 07523v3 Announce Type: replace Abstract: Transferring knowledge by fine-tuning large-scale pre-trained networks has become a standard paradigm for downstream tasks, yet the knowledge of a pre-trained model is tightly coupled with monolithic architecture, which restricts flexible reuse across models of varying scales.
arXiv:2610.07008v1 Announce Type: new Abstract: Few-shot class-incremental learning (FSCIL) aims to learn novel classes from limited annotations while preserving prior knowledge. Existing methods typ...
arXiv:2510. 24616v4 Announce Type: replace-cross Abstract: For four decades statistical physics has been providing a framework to analyse neural networks.
The paper presents a scalable method for learning dataweights that optimizes training-example importance using metagradients, guided by a small high‑quality target dataset. It introduces parametric weighting rules that generalize to new and streaming data and can transfer from smaller to larger models. Applied to reward modeling, the learned weights improve held‑out performance, adapt models to specialized domains, and enable human‑synthetic supervision mixtures to outperform either source alone.
Physics-Transfer Learning (PT) is a physics-constrained learning theory that enables zero-shot generalization across domains of increasing complexity. PT builds chains of anchor domains (CADs) linking low-cost source domains to complex targets and introduces computable feasibility and reliability indicators to assess transfer before target labels are available. The authors derive an upper bound on target risk under physical-consistency and stable-representation assumptions, and demonstrate PT’s effectiveness on quantum excited-state energy prediction and classical dynamical-transition prediction, achieving accurate high-fidelity predictions across substantial distribution shifts.
Large neural networks often acquire capabilities that small models fail to learn. Does this stem from large models learning more representative features, or from being more robust to unaccounted-for a...
arXiv:2607. 02850v1 Announce Type: new Abstract: Meta-learning without labeled data is crucial for real-world applications, where obtaining labeled datasets can be expensive or restricted due to privacy concerns.
The paper presents a tight generalization bound for deterministic neural networks that relies on the quality of their last hidden layer representations rather than traditional parameter-space metrics. By using a parameter‑free k‑nearest‑neighbor classifier to assess representation quality, the bound decomposes into three measurable components—local class geometry, decoder‑network agreement, and training stability—providing both a non‑vacuous certificate and a diagnostic tool for understanding generalization. Experiments show the bound achieves an 8.0‑percentage‑point gap to test error on CIFAR‑10 with ResNet‑18, yields the first non‑vacuous certificates on CIFAR‑100, and certifies 1.4 % on CIFAR‑10 and 23.1 % on 1,000‑class ImageNet using frozen DINOv2 representations.
arXiv:2609.39512v1 Announce Type: new Abstract: The small-sample learning problem remains a fundamental challenge in machine learning because limited training data lead to unstable model estimation a...
The paper investigates why large neural networks outperform smaller ones, focusing on a phenomenon called primacy bias—where early data distributions hinder learning of later ones. It shows that overparameterized models are more robust to this bias, especially during sequential pretraining on heterogeneous data. The authors propose Exposure Therapy, a regularization technique that improves small models’ performance on late data and overall capabilities up to the billion‑parameter scale.
arXiv:2607. 12464v1 Announce Type: cross Abstract: When labeled data are scarce, off-the-shelf diffusion models can augment training sets for few-shot medical image classification, but not all generated samples are equally useful for the downstream task.