arXiv AI
Jun 2
Domain-Shift-Aware Conformal Prediction for Large Language Models
arXiv:2510. 05566v2 Announce Type: replace-cross Abstract: Large language models have achieved impressive performance across diverse tasks.
Bridging Domain Expertise and Generalization for Performance Estimation
arXiv:2606. 06335v1 Announce Type: new Abstract: Performance estimation under distribution shift aims to predict how a model behaves on an unlabeled test set whose distribution differs from the training data, a scenario that requires reliable indicators that can faithfully reflect model behavior without ground-truth labels.
Related stories
arXiv Machine Learning
Jul 24
DataPrep-Bench: Benchmarking LLMs as Training Data Preparators
arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.
arXiv Machine Learning
Aug 4
Learning the Pareto Frontier of Predictive Models under Distribution Shift
arXiv:2608. 00632v1 Announce Type: new Abstract: Modern machine learning pipelines increasingly rely on reusing pretrained and foundation models across downstream tasks.
arXiv Machine Learning
Jul 10
Prediction-Powered Active Testing
arXiv:2607. 08347v1 Announce Type: cross Abstract: Active testing provides a label--efficient approach to risk estimation by adaptively selecting which test points should be labelled.
arXiv AI
Jun 12
Cross-Model Disagreement as a Label-Free Correctness Signal
arXiv:2603. 25450v2 Announce Type: replace Abstract: Detecting when a language model is wrong without ground truth labels is a fundamental challenge for safe deployment.
arXiv Machine Learning
Jul 21
LFM: Leveraging Foundation Models for Source-Free Universal Domain Adaptation
arXiv:2607. 17653v1 Announce Type: cross Abstract: Source-free universal domain adaptation (SF-UniDA) adapts a pre-trained source model to an unlabeled target domain under both covariate and label shifts, without access to source data.
arXiv Machine Learning
Jul 22
How Benchmark Prediction from Fewer Data Misses the Mark
arXiv:2506. 07673v2 Announce Type: replace Abstract: Large language model (LLM) evaluation is increasingly costly, prompting interest in methods that speed up evaluation by shrinking benchmark datasets.
arXiv AI
Aug 5
Efficient unsupervised domain adaptation via self-supervised vision transformer and synergistic cross-domain alignment
arXiv:2407. 21311v2 Announce Type: replace-cross Abstract: Unsupervised domain adaptation (UDA) aims to mitigate domain shift, where the distribution of labeled source data differs from that of unlabeled target data.
arXiv AI
Jul 3
Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging
arXiv:2510. 17426v3 Announce Type: replace-cross Abstract: The "alignment tax" of post-training is typically framed as a drop in task accuracy.
arXiv Machine Learning
Jun 30
Bandwidth Selection in Kernel Density Estimation for Model Calibration
arXiv:2606. 29925v1 Announce Type: new Abstract: As deep learning models are increasingly deployed in high-stakes applications, providing well-calibrated uncertainty estimates has become as critical as achieving high predictive accuracy.
arXiv AI
Jun 11
To Intervene or Not: Guiding Inference-time Alignment with Probabilistic Model Blending
arXiv:2606. 11201v1 Announce Type: cross Abstract: The wide deployment of LLMs has made model alignment necessary to make newly trained models safely and effectively respond to user instructions.
arXiv AI
Jul 3
Do Physics Foundation Models Learn Generalizable Physics? A Bias-Aware Benchmark Across Physical Regimes and Distribution Shifts
arXiv:2605. 29283v2 Announce Type: replace-cross Abstract: Recent physics foundation models claim general spatiotemporal forecasting ability, yet their evaluations often collapse performance into a single average score under a fixed training distribution.