arXiv AI
Aug 12
The Matching Principle: When Does a Training Penalty Cover Deployment Shift?
arXiv:2605. 22800v3 Announce Type: replace-cross Abstract: Ordinary training optimises the task loss and then stops.
Supervised Learning Has a Geometric Blind Spot
arXiv:2604. 21395v3 Announce Type: replace-cross Abstract: Ordinary supervised training minimises the task loss and then stops.
Related stories
arXiv Machine Learning
Jul 21
The Label Complexity of Class-Conditional Coverage under Distribution Shift
arXiv:2607. 18088v1 Announce Type: new Abstract: Standard evaluation of many recognition systems contains distribution shift by construction, since benchmarks place disjoint conditions in the training and test splits.
arXiv Machine Learning
Sep 24
Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces
arXiv:2609.27988v1 Announce Type: cross Abstract: Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direc...
arXiv Machine Learning
Sep 23
Margin-Drop Coordinates for Cross-Budget Robustness Evaluation
arXiv:2609.26081v1 Announce Type: new Abstract: Fixed-budget robustness evaluation can select the wrong frozen vision encoder. An encoder that survives a shallow attack may lose most of that robustne...
arXiv AI
Sep 18
Information-Geometric Inverse Distillation for Enhancing Adversarial Transferability
The paper introduces Inverse Knowledge Distillation (IKD), an attack‑agnostic technique that enhances adversarial transferability by maximizing the discrepancy between benign and adversarial prediction distributions on a surrogate model. IKD employs a CE/KL‑equivalent soft‑label objective to push adversarial predictions away from a fixed benign anchor, leveraging Fisher‑sensitive surrogate directions. The authors provide theoretical analysis showing CE and KL induce identical gradients, derive a lower bound on Fisher‑subspace overlap, and demonstrate through extensive ImageNet experiments that IKD consistently improves black‑box attack performance across CNN, ViT, and defended models.
Hugging Face Trending Papers
Sep 23
Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces
Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason...
arXiv Machine Learning
Jul 3
Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity
arXiv:2511. 02496v2 Announce Type: replace Abstract: We study latent geometry as an explicit component of representation quality in data-scarce learning.
arXiv Machine Learning
2d ago
When a Flatness Proxy Is Not a Function: Robustness Certificates and Training Interventions
arXiv:2609.38540v1 Announce Type: new Abstract: A valid curvature upper bound need not justify either a robustness certificate or an intervention on an intrinsic predictor property. We demonstrate th...
arXiv AI
Jun 18
Generalized Kullback-Leibler Divergence Loss
arXiv:2503. 08038v2 Announce Type: replace-cross Abstract: In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of (1) a weighted Mean Square Error (wMSE) loss and (2) a Cross-Entropy loss incorporating soft labels.
arXiv AI
6d ago
Teacher-Anchored Selection of Post-Training Quantized Models under Domain Shift
The paper investigates how to choose the best quantized model from a family of compressed versions when target labels are scarce or unavailable. It finds that a simple rule based on minimum teacher distortion consistently selects the same eight‑bit, per‑channel, unclipped configuration, though this does not minimize empirical target cross‑entropy. The study also shows that confidence‑based estimators perform poorly in overconfident regimes, while output‑distribution estimators can outperform the teacher in some architectures, and that combining distortion with a supervised term can improve selection. Across 134 candidate families, teacher‑anchored selection reduces mean regret with very few labels, though the benefit diminishes after about 25 labels.
arXiv Machine Learning
Sep 21
On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisation
The paper investigates the limits of the maximal coding rate reduction (MCR²) framework for out‑of‑distribution (OOD) generalisation. It shows that MCR² can lead to complete prediction failure under distribution shift, even when a perfectly stable feature is available, and that adding invariance principles from IRM or REx does not resolve this issue. The authors conclude that additional assumptions or learning principles are needed to guarantee stable OOD predictions with MCR².
arXiv Machine Learning
Aug 26
Focal Calibration Loss: Controlling Posterior Distortion in Deep Neural Classifiers
arXiv:2410.18321v3 Announce Type: replace Abstract: Confidence calibration matters wherever a classifier's probabilities, not just its labels, are consumed downstream. We study Focal Calibration Loss...