arXiv AI

ICON Decomposition: Auditing deep neural networks for shortcuts by decomposing layer-wise representations using concepts

The paper introduces ICON Decomposition, a method for auditing deep neural networks by decomposing layer-wise representations into independent concept contributions. Unlike existing techniques that rely on linear probes or concept activation vectors, ICON quantifies the variance share each concept explains while conditioning on all other concepts and the outcome, allowing comparison across layers and concept types. Experiments on simulated data, skin‑cancer, and neuroimaging models show that ICON more accurately recovers true concept importance and can distinguish learned shortcuts from correlated concepts, as validated by retraining and out‑of‑distribution tests.

arXiv AI

Oct 16

ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on

The paper introduces ICON decomposition, a method for auditing neural networks by decomposing the variance of intermediate representations into contributions from Independent Canonical Concepts (ICON). Unlike traditional decodability metrics, ICON isolates each concept’s influence while conditioning on all other concepts and the outcome, providing a clearer view of which concepts truly drive predictions. Experiments on simulated data and real-world skin‑cancer and neuroimaging tasks show ICON outperforms seven baseline methods and successfully distinguishes genuine shortcuts from merely correlated features.

By Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz, Nys Tjade Siegel, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, Sonja Greven, Kerstin Ritter

arXiv AI

Sep 1

ICON Decomposition: Auditing Deep Neural Networks with Multivariate Variance-based Concept-level Explanations

arXiv:2608.26083v2 Announce Type: replace-cross Abstract: Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Auditing for shortcuts requires testing many...

By Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz, Nys Tjade Siegel, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, Sonja Greven, Kerstin Ritter

arXiv Machine Learning

Aug 27

ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing

ICON Decomposition is a new method for explaining deep neural networks by quantifying how much variance each concept explains in a network layer after accounting for all other concepts and the outcome. Unlike previous concept‑based methods that evaluate concepts in isolation, ICON can distinguish genuine model reliance from spurious correlations. Experiments on synthetic data, skin‑lesion, and brain‑imaging models show that ICON recovers concept importance more accurately, isolates truly relied‑upon concepts, and provides sparse explanations validated through retraining and out‑of‑distribution testing.

By Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz, Nys Tjade Siegel, Maximilian Dreyer, Frederik Pahde, Wojciech Samek, Sonja Greven, Kerstin Ritter

arXiv AI

Jun 2

Suppressing Forgery-Specific Shortcuts for Generalizable Deepfake Detection

arXiv:2606. 01843v1 Announce Type: cross Abstract: Deepfake detection suffers from poor generalization across forgery methods, as existing models tend to rely on spurious method-specific shortcuts that fail to transfer to unseen manipulations.

By Yihui Wang, Yonghui Yang, Jilong Liu, Fengbin Zhu, Le Wu, Tat-Seng Chua

benchmarks safety

arXiv Machine Learning

Aug 12

Concept Labels Are Not Enough: Rethinking Concept Bottleneck Models through Representation Integrity

arXiv:2510. 15770v4 Announce Type: replace-cross Abstract: Although deep neural networks achieve strong predictive performance, their internal reasoning often remains difficult to inspect and control.

By Gaoxiang Huang, Songning Lai, Yutao Yue

arXiv AI

Jul 29

Analysis of the Shortcut Learning and Clever Hans Effect in CNN based ECG Image Classification

arXiv:2607. 25117v1 Announce Type: cross Abstract: Deep learning models for ECG image classification may achieve high accuracy by exploiting non-physiological visual cues instead of ECG waveform morphology.

By Abhay Kumar Pathak, Mrityunjay Chaubey, Manjari Gupta, Deepti Mishra

computer-vision safety

arXiv Machine Learning

Jun 24

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination

arXiv:2311. 02960v5 Announce Type: replace Abstract: Over the past decade, deep learning has proven to be a highly effective tool for learning meaningful features from raw data.

By Peng Wang, Xiao Li, Can Yaras, Zhihui Zhu, Laura Balzano, Wei Hu, Qing Qu

fine-tuning

arXiv Computer Vision

Oct 1

Aperture: Training-Free Multiscale Concept Bottlenecks for Remote Sensing

arXiv:2609.38603v1 Announce Type: new Abstract: While earth observation models have advanced substantially, they still lack interpretability. While concept-bottleneck models provide interpretability...

By Rishabh Mondal, Nipun Batra, Utkarsh Mall

llms safety

arXiv AI

Jul 7

TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Models

arXiv:2607. 04593v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated impressive capabilities across different tasks, but their computational cost is dominated by the large number of visual tokens fed to the language model.

By Riccardo Renzulli, Gabriele Spadaro, Shruthi Gowda, Alaa Eddine Mazouz, Van-Tam Nguyen

llms diffusion fine-tuning efficiency multimodal benchmarks

arXiv Machine Learning

Jun 16

Assessing Reliability of Symbol Detection in Concept Bottleneck Models

arXiv:2606. 16535v1 Announce Type: new Abstract: Concept Bottleneck Models (CBMs) are a relevant tool for explainable Artificial Intelligence because they make their predictions through human-interpretable symbols.

By Javier Fumanal-Idocin, Javier Andreu-Perez

arXiv Machine Learning

Aug 7

Invisible Shortcuts: Why Vision Encoders Know Your Camera

arXiv:2608. 05424v1 Announce Type: cross Abstract: Deep vision models exploit shortcuts, relying on cues that correlate with supervision signals.

By Vladan Stojni\'c, Ryan Ramos, Giorgos Kordopatis-Zilos, Noa Garcia, Giorgos Tolias

arXiv Machine Learning

Jun 19

Multimodal Concept Bottleneck Models

arXiv:2606. 19882v1 Announce Type: cross Abstract: Concept Bottleneck Models (CBMs) enhance the interpretability of deep learning networks by aligning the features extracted from images with natural concepts.

By Tongqing Shi, Ge Yan, Tuomas Oikarinen, Tsui-Wei Weng

rag multimodal benchmarks safety

Related stories