arXiv Machine Learning
Aug 4
Information-Theoretic Foundations for Machine Learning
arXiv:2407. 12288v5 Announce Type: replace-cross Abstract: The progress of machine learning over the past decade is undeniable.
A Theory of the Mechanics of Information: Generalization Through Measurement of Uncertainty (Learning is Measuring)
The paper proposes a model‑free, instance‑based learning framework that uses surprisal—an information‑theoretic measure of uncertainty—to analyze raw data directly, bypassing explicit distribution modeling. By quantifying relevance through uncertainty, the method supports generalizable inference across diverse tasks such as generative modeling, causal discovery, anomaly detection, and time‑series forecasting, while offering traceability and interpretability. The authors claim state‑of‑the‑art performance on anomaly detection and some supervised tasks, and argue that the mathematical foundations create a ‘physics’ of information that can underpin scalable, human‑understandable AI.
Related stories
arXiv Machine Learning
Jul 28
Forgetting is Everywhere
arXiv:2511. 04666v4 Announce Type: replace Abstract: A fundamental challenge in developing general learning algorithms is their tendency to forget past knowledge as they adapt to new data.
arXiv Machine Learning
Sep 15
Beyond Point Forecasts: A Survey on Probabilistic Forecasting for Time Series and Spatiotemporal Data
arXiv:2609.13345v1 Announce Type: cross Abstract: Probabilistic forecasting is central to decision-making under uncertainty, yet its methodological landscape has become increasingly fragmented across...
arXiv Machine Learning
Jun 19
Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence
arXiv:2606. 19353v1 Announce Type: cross Abstract: In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations.
arXiv Machine Learning
Jul 7
An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression
arXiv:2504. 18433v3 Announce Type: replace Abstract: Uncertainty quantification is crucial in machine learning, yet most (axiomatic) studies of uncertainty measures focus on classification, leaving a gap in regression settings with limited formal justification and evaluations.
arXiv Machine Learning
5d ago
Adaptive Uncertainty Quantification on Simplicial Complexes with Conformalized Simplicial Neural Networks
The paper introduces CF‑SNN, a conformalized simplicial neural network framework that provides uncertainty quantification for predictions on simplicial complexes. It generates simultaneous prediction sets for multiple targets, guaranteeing that each set contains the true label with a user‑specified coverage probability. Experiments across diverse SC prediction tasks show that CF‑SNN achieves the desired coverage while keeping prediction sets small and informative.
arXiv Machine Learning
Sep 15
Machine Learning under Imperfect Data: Challenges and Methods
arXiv:2609.13914v1 Announce Type: new Abstract: Machine-learning models are commonly developed under an assumption that training and test data are sufficiently complete, balanced, labelled, and drawn...
arXiv Machine Learning
Oct 7
A Data-Driven Framework for Unsupervised Monitoring of Transmission Systems Using End-of-Line Testing Data: A Case Study at Ford Motor Company
The paper presents a two‑stage multivariate monitoring framework for high‑dimensional, temporally dependent data from end‑of‑line testing in automotive manufacturing. Stage one cleans and aligns the data, while stage two applies nonlinear dimensionality reduction and a control‑chart‑based anomaly detection procedure that can operate in unsupervised or supervised modes. Applied to Ford Motor Company production data, the framework outperforms the existing model, raising accuracy from 0.50 to 0.625, recall from 0.30 to 1.00, and F1 score from 0.429 to 0.769.
arXiv Machine Learning
Sep 24
A Discrepancy-Based Perspective on Dataset Condensation
The paper introduces a unified framework for dataset condensation (DC) that generalizes existing methods by using discrepancy measures to quantify the distance between probability distributions. It extends the traditional goal of DC—creating a small synthetic dataset that preserves generalization—to include additional objectives such as robustness and privacy. The framework positions DC as a formal approximation problem, broadening its applicability across different machine learning regimes.
arXiv AI
Jul 22
RAMP: Recognition parametrisation by Amortised Message Passing
arXiv:2607. 18883v1 Announce Type: cross Abstract: A central aim of unsupervised learning is to uncover latent factors that explain dependencies among observations.
arXiv AI
Jul 7
Machine Unlearning via Information Theoretic Regularization
arXiv:2502. 05684v5 Announce Type: replace-cross Abstract: How can we effectively remove or ``unlearn'' undesirable information, such as specific features or the influence of individual data points, from a learning outcome while minimizing utility loss and ensuring rigorous guarantees?
arXiv AI
Jun 12
Existence Precedes Value: Joint Modeling of Observational Existence and Evolving States in Time Series Forecasting
arXiv:2606. 13571v1 Announce Type: cross Abstract: Real-world time series are often highly incomplete and irregular due to sensor dormancy, transmission delays, and event-driven sampling, making reliable forecasting fundamentally challenging.