arXiv:2611.07753v1 Announce Type: new Abstract: Neural collapse (NC) describes the elegant geometric structure that sample features and classifier vectors exhibit when a classification model's loss r...
arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.
The paper introduces LIDFS, an unsupervised feature selection technique that uses Local Intrinsic Dimensionality (LID) to rank features based on their local discriminability. By treating each feature as a point and estimating its LID via maximum likelihood from distances to its k nearest neighboring features, LIDFS assigns higher importance to features with lower LID and refines the ranking to remove locally redundant features. Experiments on 23 high‑dimensional datasets show that LIDFS outperforms both classic and state‑of‑the‑art methods in supervised and clustering evaluations while running faster than its closest competitor.
The paper presents a tight generalization bound for deterministic neural networks that relies on the quality of their last hidden layer representations rather than traditional parameter-space metrics. By using a parameter‑free k‑nearest‑neighbor classifier to assess representation quality, the bound decomposes into three measurable components—local class geometry, decoder‑network agreement, and training stability—providing both a non‑vacuous certificate and a diagnostic tool for understanding generalization. Experiments show the bound achieves an 8.0‑percentage‑point gap to test error on CIFAR‑10 with ResNet‑18, yields the first non‑vacuous certificates on CIFAR‑100, and certifies 1.4 % on CIFAR‑10 and 23.1 % on 1,000‑class ImageNet using frozen DINOv2 representations.
arXiv:2605.11608v2 Announce Type: replace-cross Abstract: A single base LLM now comes with dozens of post-training variants, quantized, LoRA-adapted, or distilled, and each has to be checked before r...
arXiv:2610.27443v1 Announce Type: new Abstract: Set-valued classification (SVC) offers a practical approach to communicating uncertainty while maintaining accuracy by predicting variable-sized sets p...
The paper demonstrates that neural networks can maintain perfect training accuracy while their training cross‑entropy remains nearly flat at a positive level, a phenomenon observed on modular addition tasks. Using ReLU networks with supported‑Polar hidden updates, normalized output‑layer updates, and weight decay, the authors show that hidden features evolve and held‑out accuracy can rise from zero to above 90% even as cross‑entropy stays flat. Theoretical results for one‑hidden‑layer networks and empirical evidence from deeper networks and transformers confirm that correct classification need not coincide with minimal cross‑entropy, challenging conventional assumptions about loss plateaus.
arXiv:2610.27029v1 Announce Type: new Abstract: The feature-extraction quality of pre-trained models sets the ceiling of downstream transfer learning, yet few-shot model selection remains difficult:...
arXiv:2607. 10391v1 Announce Type: cross Abstract: Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is considered for downstream tasks.
arXiv:2608. 09091v1 Announce Type: cross Abstract: Transfer learning is particularly useful in settings with limited training data, and within image classification it is common to transfer learn upon massive datasets like ImageNet , CIFAR-100, or COCO .
arXiv:2607. 12916v1 Announce Type: new Abstract: In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations.
In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations. The proposed loss encourages intra-class collapse and inter-class contrast while preserving sufficient flexibility for neural networks to approximate geometrically optimal embeddings with large angular separation between classes.