arXiv:2608. 04049v1 Announce Type: cross Abstract: The principle of Occam's razor, which instructs us to prefer simplicity in inductive inference, has attracted much scrutiny both in the philosophy of science and in machine learning.
arXiv:2608. 03386v1 Announce Type: new Abstract: Contemporary deep learning methods generalize well even when they fit their training data perfectly, a phenomenon known as benign interpolation.
The paper titled "Pessimistic Meta-Induction and Its Limits: Lessons from Frequentist Statistics and Machine Learning Theory" critiques the pessimistic meta-inductive argument against scientific realism by attacking its inductive step rather than its historical premise. It introduces a new challenge, drawing on frequentist statistics, machine learning, and formal epistemology to assess induction through convergence to truth. The authors argue that ordinary enumerative induction can achieve convergence everywhere, whereas meta-induction fails to achieve even almost everywhere convergence, and in contexts where meta-induction applies, no inference method can achieve almost everywhere convergence.
The paper investigates fundamental limits of algorithmic principles in multiclass learning, specifically proper learning and regularization. It shows that learning cannot always be reduced to proper learning even with an enlarged hypothesis class, that proper learners may need a sublinear number of errors that can be arbitrarily large, and that regularization (SRM or local) is not universally sufficient. The authors also provide a positive theory giving sufficient conditions for SRM learnability and a characterization via integrability of revealed preferences.
arXiv:2608.30246v1 Announce Type: cross Abstract: The fundamental theorem of statistical learning states that, under suitable measurability assumptions, finite Vapnik--Chervonenkis (VC) dimension gua...
arXiv:2407. 12288v5 Announce Type: replace-cross Abstract: The progress of machine learning over the past decade is undeniable.
The paper presents an order-theoretic characterization of consistent inductive inference for arbitrary binary hypothesis classes. It shows that consistency—making only finitely many prediction errors on any infinite sequence labeled by an unknown hypothesis—can be captured by a single linear order on finite realizable traces. This order must satisfy two conditions: conflicting traces select different least subtraces, and the order is well‑founded on traces of each fixed target, enabling a learner whose evidence decreases with each mistake. Conversely, any consistent learner induces such an order via canonical mistake transcripts and the Kleene–Brouwer ordering.
The paper investigates regression with bounded responses, comparing two learning frameworks: model selection aggregation, which requires improper algorithms to achieve minimax excess risk, and universal learning, where empirical risk minimization suffices for exponential learning rates. For finite hypothesis classes, the authors show that the $Q$-aggregation estimator simultaneously attains minimax optimal tails and exponential universal rates, while other common estimators fail to do so. For countably infinite classes, they prove an inherent trade‑off between exponential universal and minimax uniform rates, resolved by combining optimal algorithms from each framework via $Q$-aggregation.
arXiv:2505. 21423v3 Announce Type: replace Abstract: The remarkable generalization properties of overparameterized networks are often attributed to implicit biases, such as norm minimization at small learning rates and low sharpness in the Edge-of-Stability regime.
arXiv:2602. 06837v2 Announce Type: replace Abstract: Hybrid modeling, the combination of machine learning models and scientific mathematical models, enables flexible and robust data-driven prediction with partial interpretability.
arXiv:2609.08961v1 Announce Type: cross Abstract: For a finite set $O$ of Boolean functions, we consider the class of propositional formulas built using the functions in $O$ as connectives. We determ...
arXiv:2602. 20971v3 Announce Type: replace-cross Abstract: Bubeck and Selke (2021) propose the connection between the Law of Robustness and robust generalization error as an open problem.
arXiv:2609.31101v1 Announce Type: cross Abstract: The flatness of the loss landscape at a minimizer is a widely used heuristic for reasoning about neural-network generalization, yet evidence for this...