arXiv:2607. 08605v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have emerged as a promising technique for mechanistic interpretability by learning a set of sparse latent features in large models, each of which encodes a distinct concept.
VLMProbe is an open‑source library that unifies lens readouts, attention attribution, causal interventions, and sparse autoencoder analysis for vision‑language models. It offers shared adapters, calibration tools, evaluation pipelines, and visualizations to enable systematic comparison across different model architectures. The framework demonstrates that representation readouts and spatial explanations vary with the analysis goal and observation point, and that local attribution can conflict with intervention outcomes, highlighting the need for multiple evidence sources in interpretation studies.
UVU is a vision-language unified autoregressive framework that integrates visual supervision directly into the pre-training stage of multimodal large language models. By using continuous visual encoding and a large-scale iterative hierarchical clustering algorithm to build a pixel-level visual codebook, UVU enables lossless representation of visual inputs and autoregressive generation of pixel-level image tokens alongside textual tokens. This approach synergizes pixel-level visual perception with semantic-level visual understanding, allowing models to internalize visual reconstruction capabilities and improve multimodal understanding performance.
arXiv:2606. 16193v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representations remain difficult to interpret.
D‑Scope is a framework that links the interpretation of sparse autoencoder (SAE) features in diffusion transformers (DiTs) to controllable image generation. It aggregates SigLIP‑2 embeddings of highly activating image patches into visual centroids, matches target text descriptions against these centroids, and retrieves individual features without per‑feature text annotations. The method provides visual evidence for each selection and uses spatially masked interventions to test decoder directions under fixed generation conditions, evaluated across 150 SAEs and a benchmark of 100 target concepts.
arXiv:2606.16193v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representat...
The paper investigates how vision transformers (ViTs) represent and process visual information by conducting a systematic analysis of 6.6K features across all layers using sparse autoencoders. It introduces the residual replacement model, which substitutes ViT computations with interpretable features in the residual stream, revealing a progression from low-level patterns to high-level semantics and specialized encoding of curves and spatial positions. This model provides a scalable, faithful, and parsimonious circuit that simplifies ViT computations, enabling intuitive understanding of ViT mechanisms and demonstrating its usefulness for debiasing spurious correlations.
arXiv:2606. 31699v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have recently been proposed as interpretable tools for concept-level manipulation, under the assumption that isolated features can serve as controllable intervention points.
WithinVAE is a compact variational visual autoencoder that learns semantic structure internally using a sequential bottleneck and self‑distillation from an exponential‑moving‑average copy of itself. It achieves a four‑fold compression ratio on ImageNet‑1K, improving reconstruction metrics such as PSNR by 1.36 dB over RPiAE while using half the latent capacity. The model also boosts linear probing performance, demonstrating that semantic gains arise from self‑distillation rather than external supervision.
arXiv:2606. 24716v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence.
arXiv:2607. 04593v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated impressive capabilities across different tasks, but their computational cost is dominated by the large number of visual tokens fed to the language model.
arXiv:2606. 15468v1 Announce Type: cross Abstract: Vision models can achieve strong performance on classification tasks, but the internal representations supporting their predictions are often difficult to interpret.