arXiv Machine Learning

Observational Multiplicity

The paper introduces the concept of observational multiplicity, where multiple probabilistic classifiers can perform similarly yet produce conflicting predictions, undermining interpretability and safety. It proposes measuring this arbitrariness through a regret metric that captures how predictions could shift with different training labels. The authors present a general method to estimate regret, show it varies across dataset groups, and discuss its use for safety via abstention and targeted data collection.

arXiv Machine Learning

Sep 3

Simulating Classification Models for Ex-Ante Evaluation of Predict-Then-Optimize Methods

The paper extends ex‑ante evaluation of Predict‑Then‑Optimize methods from binary to multiclass classification by simulating predictions at specified performance levels and mapping prediction errors to decision regret. It introduces a first‑order approximation that estimates regret from individual misclassifications, reducing computational effort. Experiments show the simulation accurately reproduces target performance and that the approximation is close for some problems, though it falters when simultaneous misclassifications interact significantly.

By Pieter Smet

arXiv Machine Learning

Jun 2

Model Multiplicity and Predictive Arbitrariness in Recidivism Risk Assessment

arXiv:2606. 02198v1 Announce Type: new Abstract: Prediction tasks over individual futures, which are inherently noisy, often admit multiple similarly accurate models.

By Ashwin Singh, Carlos Castillo

arXiv Statistics ML

Sep 10

A Unifying Perspective on Probabilities as Model Predictions

arXiv:2609.09855v1 Announce Type: new Abstract: Although probabilistic statements are ubiquitous, foundational disagreements persist about their understanding, as exemplified by debates between Bayes...

By Benedikt H\"oltgen

arXiv AI

Sep 2

Measuring consistency via ensemble margin and local prediction variability: Auditing decision systems in the presence of predictive multiplicity

The paper introduces a new consistency criterion for auditing decision systems that combines ensemble margin with local prediction variability to address predictive multiplicity, or the Rashomon effect. It shows that finite ensembles converge to the expected model’s consistency score as ensemble size and sample count grow, and demonstrates that ensembling models from the Rashomon set reduces unchecked incorrect predictions while keeping diversions moderate. Experiments on transformer and fine‑tuned language models for NLP and tabular classification confirm the method’s effectiveness and stronger alignment with existing multiplicity metrics.

By Sinjini Banerjee, Tim Marrinan, Anand D. Sarwate

llms fine-tuning

arXiv AI

Jun 9

Performative Learning Theory

arXiv:2602. 04402v3 Announce Type: replace-cross Abstract: Performative predictions influence the very outcomes they aim to forecast.

By Julian Rodemann, Unai Fischer-Abaigar, James Bailie, Krikamol Muandet

rag

arXiv Machine Learning

3d ago

Stabilizing Performative Feedback Loops with Minimal Model Deployments

arXiv:2609.14065v1 Announce Type: new Abstract: When algorithmic predictions inform people's decisions, the models we deploy are performative and actively shape the data we see. This feedback loop be...

By Gabriele Farina, Juan Carlos Perdomo

arXiv Machine Learning

Jun 5

The Stability of Online Algorithms in Performative Prediction

arXiv:2602. 24207v2 Announce Type: replace Abstract: The use of algorithmic predictions in decision-making leads to a feedback loop where the models we deploy actively influence the data distributions we see, and later use to retrain on.

By Gabriele Farina, Juan Carlos Perdomo

arXiv Machine Learning

Jun 5

Conformal Risk-Averse Decision Making with Action Conditional Guarantee

arXiv:2606. 05551v1 Announce Type: cross Abstract: Reliable decision making pipelines powered by machine learning models require uncertainty quantification (UQ) methods that come with explicit safety guarantees.

By Zihan Zhu, Shayan Kiyani, George Pappas. Hamed Hassani

safety

arXiv Machine Learning

Jun 10

Beyond Explaining Predictions: Logic-Based Explanations for Confidence in Machine Learning Models

arXiv:2606. 10347v1 Announce Type: new Abstract: Machine learning is increasingly used in critical domains, where both predictions and their associated confidence levels influence important decisions.

By Vin\'icius Peixoto Chagas, Carlos Henrique Leit\~ao Cavalcante, Thiago Alves Rocha

arXiv Machine Learning

Aug 4

Information-Theoretic Foundations for Machine Learning

arXiv:2407. 12288v5 Announce Type: replace-cross Abstract: The progress of machine learning over the past decade is undeniable.

By Hong Jun Jeon, Benjamin Van Roy

arXiv Machine Learning

Jun 26

Learning from Equivalence Queries, Revisited

arXiv:2604. 04535v2 Announce Type: replace Abstract: Modern machine learning systems, such as generative models and recommendation systems, often evolve through a cycle of deployment, user interaction, and periodic model updates.

By Mark Braverman, Roi Livni, Yishay Mansour, Shay Moran, Kobbi Nissim

reinforcement-learning safety

arXiv Machine Learning

Jun 3

Hierarchies of Calibration: Classification meets Regression

arXiv:2606. 03245v1 Announce Type: cross Abstract: Concepts of calibration formalize the compatibility between probabilistic predictions and the respective outcomes.

By Johannes Resin, Lu Yang, Tilmann Gneiting

Related stories