arXiv Machine Learning
Jul 10
Prediction-Powered Active Testing
arXiv:2607. 08347v1 Announce Type: cross Abstract: Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled.
Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts
arXiv:2602. 02229v2 Announce Type: replace Abstract: We study the problem of monitoring model performance in dynamic environments where labeled data are limited.
Related stories
Hugging Face Trending Papers
Jul 9
Prediction-Powered Active Testing
Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled. However, existing estimators fail to exploit the informative predictions of powerful black--box models, even though such predictions are increasingly available in settings where labels remain expensive.
arXiv AI
2d ago
Regime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety Classifiers
arXiv:2608. 14089v1 Announce Type: new Abstract: Safety classifiers deployed with large language models often fail for two reasons: their decisions reflect the policy learned during training rather than the deployer's desired policy, and their performance degrades as deployment traffic evolves.
arXiv Machine Learning
Jun 15
Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias
arXiv:2606. 14506v1 Announce Type: cross Abstract: Understanding how a prediction model will perform in a new environment before deployment is essential to preventing harm when algorithms inform decision-making.
arXiv Machine Learning
Jun 3
BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks
arXiv:2606. 02947v1 Announce Type: new Abstract: Supervised fine-tuning is the predominant approach for adapting autoregressive vision-language models to downstream tasks.
arXiv AI
Jun 11
Estimating Tail Risks in Language Model Output Distributions
arXiv:2604. 22167v2 Announce Type: replace-cross Abstract: Language models are increasingly capable and are being rapidly deployed on a population-level scale.
arXiv Machine Learning
Jun 16
Audited Conformal Prediction for Classification under Unknown Distribution Shift
arXiv:2606. 14909v1 Announce Type: cross Abstract: We consider the problem of uncertainty quantification for a pretrained classification model deployed under unknown distribution shift.
arXiv Machine Learning
Jul 14
Monitoring the calibration of probability forecasts with an application to concept drift detection involving image classification
arXiv:2510. 25573v2 Announce Type: replace-cross Abstract: Machine learning approaches for image classification have led to impressive advances in that field.
arXiv AI
Jul 8
Statistical Adversaries: Natural Backdoor-like Features in Vision Datasets
arXiv:2607. 05516v1 Announce Type: cross Abstract: Model-specific adversarial attacks have been extensively studied.
arXiv AI
Jul 23
Sound Probabilistic Safety Bounds for Large Language Models
arXiv:2607. 20286v1 Announce Type: cross Abstract: We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt.
arXiv Machine Learning
Jun 8
Certified Robustness to Data Poisoning in Gradient-Based Training
arXiv:2406. 05670v3 Announce Type: replace Abstract: Modern machine learning pipelines leverage large amounts of public data, making it infeasible to guarantee data quality and leaving models open to poisoning and backdoor attacks.
arXiv AI
1d ago
Statistical Adversaries: Natural Backdoor-like Adversarial Features in Clean Vision Datasets
arXiv:2607. 05516v2 Announce Type: replace-cross Abstract: Model-specific adversarial attacks have been extensively studied.