The paper introduces a graded distribution‑shift benchmark for evaluating uncertainty quantification (UQ) in engineering regression models. It defines a three‑rung shift ladder—distribution, soft, and hard shift—using physically meaningful variables and tests 12 UQ methods across seven public engineering datasets, including extrapolation, turbofan fault, and air‑quality scenarios. Results show conformal methods maintain high coverage under distribution shift, but coverage drops significantly under hard shift, with no clear method outperforming others, and weighted conformalised quantile regression often abstains from predictions.
arXiv:2608.29789v1 Announce Type: cross Abstract: Uncertainty quantification from finite data is central to machine learning, optimization, and automation systems, where decisions must remain reliabl...
arXiv:2507. 08150v4 Announce Type: replace-cross Abstract: Accurate uncertainty quantification is critical for reliable predictive modeling.
arXiv:2608. 01460v1 Announce Type: new Abstract: Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample coverage guarantees under exchangeability.
arXiv:2608. 07589v1 Announce Type: cross Abstract: Predicting fatigue failure in steel components experimentally is costly because it requires testing across multiple compositions and processing conditions.
arXiv:2606. 27001v1 Announce Type: new Abstract: Quantifying the evolution of uncertainty is critical to both probabilistic forecasting and data assimilation in numerical weather prediction.
arXiv:2609.25430v1 Announce Type: new Abstract: Neural surrogates can substantially accelerate computer-aided engineering (CAE) workflows, but their use in design requires uncertainty estimates that...
arXiv:2608. 07795v1 Announce Type: cross Abstract: Prediction intervals for multi-modal regression with tabular variables, text, images, or other input sources are difficult to calibrate when those sources disagree or one is missing.
arXiv:2608. 15290v1 Announce Type: cross Abstract: The increasing availability of large and complex datasets across many scientific disciplines has led to widespread adoption of machine learning (ML) for prediction.
arXiv:2505. 08784v2 Announce Type: replace-cross Abstract: As machine learning (ML) enters high-stakes domains, trustworthy uncertainty quantification (UQ) is essential for safety.
arXiv:2606. 04164v1 Announce Type: cross Abstract: Data samples used for training often differ from those encountered during fine-tuning and deployment, and while ML models show promise, their performance remains limited when only small annotated datasets are available.
SHIFT-JEV is a procedure for large language model (LLM) decision systems that addresses domain, label‑prior, emerging‑class, and candidate‑set shifts. It combines risk scoring from uncertainty, format, and option‑consistency signals with certified abstention thresholds, drift and label‑shift monitoring, and a gain‑gated model‑update policy. Experiments on ten shifted test environments with Qwen3 and Phi‑4-mini show that fine‑tuning with KL anchors and temperature scaling reduces calibration error, while SHIFT‑JEV’s uncertainty‑based abstention and chi‑squared shift detection outperform baseline methods, achieving higher accuracy in continual adaptation scenarios.
arXiv:2609.30886v1 Announce Type: new Abstract: Conformal prediction can lose coverage when the data distribution changes after deployment. We study adaptation using labeled source data and unlabeled...