arXiv:2605.27136v2 Announce Type: replace
Abstract: Uncertainty quantification (UQ) remains a critical challenge in Large Vision Language Models (LVLMs) for reliable predictions and real-world deploy...
By Joseph Hoche, David Brellmann, Gianni Franchi
arXiv:2512.14177v4 Announce Type: replace
Abstract: Large Vision-Language Models (LVLMs) often produce plausible but unreliable outputs, making robust uncertainty estimation essential. Recent work on...
By Joseph Hoche, Andrei Bursuc, David Brellmann, Gilles Louppe, Pavel Izmailov, Angela Yao, Gianni Franchi
Deep neural networks are increasingly deployed in safety-critical domains as perception modules, where failures are often caused due to rare and under-represented scenarios. This necessitates the need to evaluate the semantic robustness of perception models; conformance of behavior to high-level requirements over real-world perceptual variability.
Vision-language-action models (VLAs) combine vision-language backbones with expressive generative action heads trained via flow matching on large-scale robotic datasets. Despite their strong empirical performance in robotic manipulation, VLAs lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable.
arXiv:2608. 10289v1 Announce Type: cross Abstract: Deep neural networks are increasingly deployed in safety-critical domains as perception modules, where failures are often caused due to rare and under-represented scenarios.
By Nusrat Jahan Mozumder, Divya Gopinath, Corina Pasareanu, Matthew Dwyer
The paper introduces FailSAE, a method that uses Sparse Autoencoders to predict failures in vision‑language models (VLMs) such as CLIP. By treating failure prediction as a classification over sparse SAE latent activations and employing a three‑stage training pipeline, the approach yields higher prediction accuracy than existing confidence‑score or auxiliary‑classifier baselines. Analysis shows that the SAE captures class‑specific concepts and reveals a shift toward ambiguous or style‑related concepts during failures, offering insights for runtime failure recovery.
By Jie Ma, Zongxi Liu, Yi Zhu