arXiv Computer Vision
From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Jul 10
When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalities
arXiv:2607. 08605v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have emerged as a promising technique for mechanistic interpretability by learning a set of sparse latent features in large models, each of which encodes a distinct concept.
arXiv AI
Oct 27
VLMProbe: An Open-source Library for VLM Probing and Interpretation
VLMProbe is an open‑source library that unifies lens readouts, attention attribution, causal interventions, and sparse autoencoder analysis for vision‑language models. It offers shared adapters, calibration tools, evaluation pipelines, and visualizations to enable systematic comparison across different model architectures. The framework demonstrates that representation readouts and spatial explanations vary with the analysis goal and observation point, and that local attribution can conflict with intervention outcomes, highlighting the need for multiple evidence sources in interpretation studies.
arXiv Computer Vision
Sep 24
UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
arXiv AI
Jun 16
Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMs
arXiv:2606. 16193v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representations remain difficult to interpret.
arXiv AI
Oct 1
D-Scope: Decomposing and Steering Diffusion Transformers with Sparse Autoencoders
D‑Scope is a framework that links the interpretation of sparse autoencoder (SAE) features in diffusion transformers (DiTs) to controllable image generation. It aggregates SigLIP‑2 embeddings of highly activating image patches into visual centroids, matches target text descriptions against these centroids, and retrieves individual features without per‑feature text annotations. The method provides visual evidence for each selection and uses spatially masked interventions to test decoder directions under fixed generation conditions, evaluated across 150 SAEs and a benchmark of 100 target concepts.
arXiv AI
Oct 7
SAE++: Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMs
arXiv:2606.16193v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representat...