Conformal prediction is a popular tool for uncertainty quantification that outputs prediction sets with finite-sample coverage guarantees. While prediction set size is commonly used as a heuristic mea...
The paper investigates whether compression explains why over‑parameterized neural networks generalize, using the minimum description length (MDL) principle. It introduces prequential regret, a computable metric from the training trajectory, and shows that in the nearly interpolating regime it upper‑bounds the description length of the trained parameters, leading to a high‑probability generalization bound in the PAC‑MDL setting. The study also connects MDL with the algorithm‑level information bottleneck by bounding encoder‑complexity correction with MDL parameter description length, and demonstrates that prequential regret correlates strongly (Pearson > 0.9) with generalization error on CIFAR‑10 and CIFAR‑100.
By Lincen Yang, Jiayang Shi, Nan Pu, Dani\"el M. Pelt, Matthijs van Leeuwen, Zhong Li, Shujian Yu
arXiv:2505. 19033v2 Announce Type: replace-cross Abstract: Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees.
By Alireza Javanmardi, Soroush H. Zargarbashi, Santo M. A. R. Thies, Willem Waegeman, Aleksandar Bojchevski, Eyke H\"ullermeier
A central question in modern learning theory is what notion of compression, if any, explains why over-parameterized neural networks generalize. Existing answers measure complexity at different levels,...
The paper investigates the evaluated Conditional Mutual Information (eCMI) within the Conditional Mutual Information (CMI) framework for learning algorithms. It demonstrates that for VC classes, an optimal PAC guarantee can be achieved by constructing a learning algorithm whose eCMI scales as O(d) in the realizable case, where d is the VC-dimension. The proposed algorithm is a randomized Majority-of-5 base learners that attains an optimal in-expectation generalization guarantee.
By Steve Hanneke, Juexiao Wang
The paper extends conformal prediction by introducing a geometric view of uncertainty through credal sets, which are families of probability distributions compatible with CP regions. It provides analytical characterisations and efficient algorithms for common credal uncertainty measures, reducing complexity from exponential to linear after sorting p-values. The authors also unify these measures with classical conformal summaries using Tsallis entropy penalties, prove an impossibility theorem for split-conformal regression, and demonstrate through experiments that the new summaries recover ranking information lost by prediction-set size, enabling finer-grained uncertainty assessment.
By Siu Lun Chau, Soroush H. Zargarbashi, Yusuf Sale, Michele Caprio