arXiv:2606. 00344v1 Announce Type: new Abstract: Neural collapse is a structural property of the last-hidden-layer activations in neural network classification models, when trained beyond a zero classification error.
arXiv:2608. 09417v2 Announce Type: replace Abstract: Deep decoder-only Transformers often replace the original Post-Norm architecture with Pre-Norm variants because Post-Norm training is highly sensitive to warmup and learning rate under conventional initialization schemes.
arXiv:2608. 09417v1 Announce Type: new Abstract: Deep decoder-only Transformers often replace the original Post-Norm architecture with Pre-Norm variants because Post-Norm training is highly sensitive to warmup and learning rate under conventional initialization schemes.
arXiv:2608.21464v1 Announce Type: cross Abstract: We demonstrate that structured distortion of training data - which we term complexity induction - can induce compositional generalization in a standa...
arXiv:2606. 11319v1 Announce Type: new Abstract: Learning from imperfect data is a central theme in machine learning, connecting practical questions of robustness to fundamental questions of learnability.
arXiv:2607. 12094v1 Announce Type: cross Abstract: Reliable detection of out-of-distribution (OOD) samples is crucial for the safe deployment of machine learning models.
arXiv:2508. 09697v4 Announce Type: replace Abstract: Noisy labels are inevitable in real-world multimedia applications.
arXiv:2511. 01938v3 Announce Type: replace-cross Abstract: Grokking is a puzzling phenomenon in neural networks where full generalization occurs only after a substantial delay following the complete memorization of the training data.
arXiv:2604. 00230v2 Announce Type: replace Abstract: Neural collapse (NC) -- the convergence of penultimate-layer features to a simplex equiangular tight frame -- is well understood at equilibrium, but the dynamics governing its onset remain poorly characterised.
arXiv:2508. 09697v3 Announce Type: replace Abstract: Noisy labels are inevitable in real-world scenarios.
arXiv:2505. 22578v2 Announce Type: replace Abstract: The optimization of neural networks under weight decay remains poorly understood from a theoretical standpoint.
The paper introduces "complexity induction," a method that distorts training data in a structured way to promote compositional generalization in a standard CNN without changing its architecture. Using synthetic images of colored geometric shapes, the authors encode classes as flat string labels (e.g., "red‑circle") and deliberately omit certain color‑shape combinations from training. Two distortion techniques—mixed labels (soft target distributions based on Jaccard similarity) and expanded dataset (false samples with incorrect labels)—both enable the model to predict unseen class combinations, with mixed labels leveraging the CNN’s embedding structure and expanded training improving embedding factorization. A control experiment with random false labels shows that the effect relies on the structured nature of the distortion rather than noise alone.
arXiv:2607. 13631v1 Announce Type: new Abstract: The Hessian matrix is an important quantity of interest when it comes to studying the loss landscape and optimization dynamics in deep learning, as well as designing measures of generalization, second-order learning algorithms, etc.