arXiv Machine Learning
Aug 4
Information-Theoretic Foundations for Machine Learning
arXiv:2407. 12288v5 Announce Type: replace-cross Abstract: The progress of machine learning over the past decade is undeniable.
Quantifying the Value of Privileged Information Using a PAC-Bayesian Approach
The paper introduces a PAC‑Bayesian, algorithm‑agnostic framework to quantify the value of privileged information (PI) in Learning Using Privileged Information (LUPI). By comparing the tightest achievable risk bounds with and without PI, the authors derive a training‑time metric that estimates the maximum potential gain from PI without requiring test data. Experiments in supervised and unsupervised settings show a strong correspondence between this metric and actual test‑time performance improvements.
Related stories
arXiv AI
Aug 18
Informative Perturbation Selection for Uncertainty-Aware Post-hoc Explanations
arXiv:2603. 14894v3 Announce Type: replace-cross Abstract: Trust and ethical concerns due to the widespread deployment of opaque machine learning (ML) models motivating the need for reliable model explanations.
arXiv Machine Learning
Jul 7
Distribution-free Deviation Bounds and The Role of Domain Knowledge in Learning via Model Selection with Cross-validation Risk Estimation
arXiv:2303. 08777v3 Announce Type: replace-cross Abstract: Cross-validation is one of the most widely used tools for risk estimation and model selection in statistics and machine learning, yet its theoretical properties when embedded in a learning procedure remain insufficiently understood.
arXiv Computation and Language
Aug 25
What Language Models Know But Don't Say: Non-Generative Prior Extraction for Generalization
arXiv:2601.17609v3 Announce Type: replace Abstract: In domains like medicine and finance, large-scale labeled data is costly and often unavailable, leading to models trained on small datasets that st...
arXiv Machine Learning
Sep 14
Explanations-Driven Active Feature Acquisition for Algorithmic Recourse
The paper introduces Explanation-Driven Feature Acquisition (EDFA), a method that jointly optimizes algorithmic recourse and feature acquisition by selecting features based on explanatory value per unit cost. Using Markov Blanket theory, EDFA unifies various explanation types and provides distribution‑free validity guarantees for recourse derived from partial information. Experiments on seven datasets show that EDFA requires fewer features than existing active feature acquisition baselines while maintaining accuracy and producing more actionable recourse.
arXiv Machine Learning
Jul 15
Reducing information dependency does not cause training data privacy. Adversarially non-robust features do
arXiv:2607. 12354v1 Announce Type: new Abstract: In this paper, we challenge the prevailing view that information dependency (including rote memorization) drives training data exposure to image reconstruction attacks.
arXiv Machine Learning
Jun 15
Private Prediction via PAC Privacy
arXiv:2601. 14033v2 Announce Type: replace Abstract: Machine learning models are increasingly served behind APIs.
arXiv Machine Learning
Aug 11
Coupled Training with Privileged Information and Unlabeled Data
arXiv:2605. 23268v2 Announce Type: replace-cross Abstract: In many prediction problems, we have extra information during training (for example, measurements that are expensive or slow to collect) that will not be available when the model is deployed.
arXiv Machine Learning
Jul 10
Prediction-Powered Active Testing
arXiv:2607. 08347v1 Announce Type: cross Abstract: Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled.
arXiv AI
Aug 11
Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models
arXiv:2512. 11614v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on retrieved context to guide large language models (LLM), yet treats the retrieval as a heuristic rather than verifiable evidence -- leading to unsupported answers, hallucinations, and reliance on spurious context.
arXiv Machine Learning
Jun 10
SoK: Colluding Adversaries in Machine Learning Pipelines
arXiv:2606. 10091v1 Announce Type: cross Abstract: Machine learning (ML) models are susceptible to various security, privacy, and fairness risks.
arXiv AI
Jul 3
DemoPSD: Disagreement-Modulated Policy Self-Distillation
arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.