← Back to all news
arXiv AI By Yilin Zhang, Han Jiang, Cai Xu, Ying Liu, Wei Zhao

Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration

Read the original on arXiv AI →

The paper introduces Calibration-Aware Uncertainty Cascades (CAUC), a post‑hoc framework that calibrates each model’s confidence independently and uses these calibrated scores to decide when to accept an early prediction, invoke a stronger model, or combine outputs. CAUC establishes a common reliability scale across heterogeneous models, decoupling deployment policies from specific model pools or budgets. Experiments on six language benchmarks show a 1.9% relative accuracy gain over strong‑model‑only inference while cutting strong‑model calls by about 47%, and on image classification it maintains or improves performance while reducing GFLOPs by up to 57%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jul 24

Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating

arXiv:2607. 20481v1 Announce Type: new Abstract: Local-cloud collaboration is a practical way to deploy large language models under resource constraints, but existing methods often rely on trained routers or collaboration-aware finetuning that tie routing behavior to a particular operating regime.

By Evan Chen, Shiqiang Wang, Kevin S Chan, Su Wang, Christopher Brinton
llmsfine-tuning
More like this →
arXiv Machine Learning
Oct 22

Estimating Performance for Safe Deployment of Machine Learning Models

The paper introduces MOVAL, a model evaluation tool that estimates performance of deep neural networks on unlabeled test data by calibrating class-wise confidence scores. MOVAL mitigates issues from distributional shifts and class imbalance, enabling estimation of metrics beyond accuracy such as F1-Score and AUC. Experiments on natural and medical image datasets demonstrate state‑of‑the‑art results across classification and segmentation tasks, providing accurate class, instance, and pixel‑level performance estimates for safe deployment.

By Zeju Li, Konstantinos Kamnitsas, Mobarakol Islam, Melanie Roschewitz, Cheng Ouyang, Chen Chen, Ben Glocker
computer-visionbenchmarkssafety
More like this →