arXiv Machine Learning

An Information-Theoretic Framework for Feature Construction in Out-of-Distribution Detection

arXiv:2506. 14194v2 Announce Type: replace Abstract: We present a theory for the construction of out-of-distribution (OOD) detection features for neural networks.

arXiv Machine Learning
Jul 15

Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation

arXiv:2505. 15284v2 Announce Type: replace Abstract: Out-of-Distribution (OoD) detection is vital for the reliability of deep neural networks, the key of which lies in effectively characterizing the disparities between OoD and In-Distribution (InD) data.

By Kun Fang, Qinghua Tao, Mingzhen He, Kexin Lv, Runze Yang, Haibo Hu, Xiaolin Huang, Jie Yang, Longbing Cao
arXiv Machine Learning
Sep 21

On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisation

The paper investigates the limits of the maximal coding rate reduction (MCR²) framework for out‑of‑distribution (OOD) generalisation. It shows that MCR² can lead to complete prediction failure under distribution shift, even when a perfectly stable feature is available, and that adding invariance principles from IRM or REx does not resolve this issue. The authors conclude that additional assumptions or learning principles are needed to guarantee stable OOD predictions with MCR².

By Menghui Zhou, Gaoshan Bi, Vitaveska Lanfranchi, Po Yang
arXiv AI
Jun 9

Investigating the Histogram Loss in Regression

arXiv:2402. 13425v3 Announce Type: replace-cross Abstract: It is becoming increasingly common in regression to train neural networks that model the entire distribution even if only the mean is required for prediction.

By Ehsan Imani, Kai Luedemann, Sam Scholnick-Hughes, Esraa Elelimy, Martha White
arXiv AI
Jul 7

Machine Unlearning via Information Theoretic Regularization

arXiv:2502. 05684v5 Announce Type: replace-cross Abstract: How can we effectively remove or ``unlearn'' undesirable information, such as specific features or the influence of individual data points, from a learning outcome while minimizing utility loss and ensuring rigorous guarantees?

By Shizhou Xu, Thomas Strohmer
arXiv Machine Learning
Sep 11

Optimizing Three Critical Factors for Practical and Effective OOD Detection Fine-Tuning

The paper introduces a framework for out-of-distribution (OOD) detection that addresses the trade‑off between detection performance and classification accuracy caused by fine‑tuning with auxiliary outlier data. It optimizes three factors—model reminder, data sampling, and representation learning—by proposing Self‑Knowledge Distillation to preserve accuracy, Semi‑hard Outlier Sampling to enhance detection with minimal data, and Outlier‑aware Supervised Contrastive Learning to improve ID‑OOD separability. The combined approach yields cumulative gains, outperforming existing methods on diverse benchmarks, especially in long‑tailed scenarios, and offers a robust baseline for real‑world OOD detection.

By Hyunjun Choi, JaeHo Chung, Hawook Jeong