arXiv Machine Learning
Visual Tripwires: Anticipating Failure in Deep Vision Systems
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computer Vision
Sep 7
FailSAE: Towards Interpretable Failure Prediction for Vision-Language Models via Sparse Autoencoders
The paper introduces FailSAE, a method that uses Sparse Autoencoders to predict failures in vision‑language models (VLMs) such as CLIP. By treating failure prediction as a classification over sparse SAE latent activations and employing a three‑stage training pipeline, the approach yields higher prediction accuracy than existing confidence‑score or auxiliary‑classifier baselines. Analysis shows that the SAE captures class‑specific concepts and reveals a shift toward ambiguous or style‑related concepts during failures, offering insights for runtime failure recovery.
arXiv AI
2d ago
Towards Reliable Vision-Language Models for Autonomous Driving
The paper evaluates five vision‑language models on autonomous driving tasks under various visual input conditions, finding that visual corruption affects accuracy and confidence differently across models and datasets. It then tests Visual Evidence Augmentation (VEA) as an inference‑time technique to enhance reliability, observing mixed improvements depending on the model and setting.
arXiv AI
Sep 16
Same Answer, Different Representations: Hidden instability in VLMs
arXiv:2602.06652v2 Announce Type: replace Abstract: The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions r...
arXiv AI
Jun 11
Task-Aligned Stability Analysis of Vision-Language Models for Autonomous Driving Hazard Detection
arXiv:2606. 11889v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used for scene understanding in autonomous driving, but robustness analysis often relies on task-agnostic embedding stability alone.
arXiv Computer Vision
Aug 24
Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions
The paper surveys Continual Test-Time Adaptation (CTTA), a framework that adapts pretrained computer‑vision models to non‑stationary target distributions without source data or labeled targets, while avoiding catastrophic forgetting and error accumulation. It formally defines the CTTA problem, categorizes existing methods into optimization‑based, parameter‑efficient, and architecture‑based families, and reviews representative techniques and benchmarks across standard evaluation settings. The survey also outlines current limitations and proposes future research directions, such as adapting foundation models and black‑box systems.
Hugging Face Trending Papers
Aug 25
Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation
End-to-end multimodal driving has progressed rapidly by fusing camera and LiDAR streams. Existing pipelines remain fragile under asymmetric sensor degradation, where either an entire modality or only...