arXiv Machine Learning

Reliable Inference in Edge-Cloud Model Cascades via Conformal Alignment

arXiv:2510. 17543v3 Announce Type: replace Abstract: Edge intelligence enables low-latency inference via compact on-device models, but assuring reliability remains challenging.

arXiv AI
Sep 12

Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration

The paper introduces Calibration-Aware Uncertainty Cascades (CAUC), a post‑hoc framework that calibrates each model’s confidence independently and uses these calibrated scores to decide when to accept an early prediction, invoke a stronger model, or combine outputs. CAUC establishes a common reliability scale across heterogeneous models, decoupling deployment policies from specific model pools or budgets. Experiments on six language benchmarks show a 1.9% relative accuracy gain over strong‑model‑only inference while cutting strong‑model calls by about 47%, and on image classification it maintains or improves performance while reducing GFLOPs by up to 57%.

By Yilin Zhang, Han Jiang, Cai Xu, Ying Liu, Wei Zhao