arXiv Machine Learning
Tree SAE: Learning Hierarchical Feature Structures in Sparse Autoencoders
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 7
Do Sparse Autoencoders Learn Meaningful Concept Hierarchies?
arXiv:2606.22994v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) have become an important tool for unsupervised concept discovery in large models. To make the resulting feature spaces m...
arXiv Machine Learning
Sep 30
When Trees Are Not Enough: Learning Mixed-Topology Feature Graphs with Adaptive Graph Sparse Autoencoders
arXiv:2609.36294v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) expose interpretable features in large language model activations, yet existing structured SAEs impose single-parent trees o...
arXiv AI
Oct 7
SAE++: Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMs
arXiv:2606.16193v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representat...
arXiv AI
Jun 16
Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMs
arXiv:2606. 16193v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language tasks, yet their internal visual representations remain difficult to interpret.
arXiv Machine Learning
Oct 16
Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility
Sparse autoencoders (SAEs) are usually trained on hundreds of millions to billions of activation tokens, but this study shows that the amount of data needed depends on the evaluation metric. Training four SAE architectures on three language models with token budgets from 5 M to 500 M revealed that reconstruction quality can reach near‑full‑data performance with far fewer tokens, while one‑to‑one feature reproducibility continues to improve as more data is used. The research also found that random sampling preserves activation distributions better for reconstruction, whereas diversity‑based sampling can enhance feature interpretability, and provided a theoretical framework distinguishing statistical consistency from the stricter conditions needed for exact feature recovery.
arXiv Machine Learning
Jul 21
Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models
arXiv:2607. 17117v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) decompose language model activations into sparse features, but standard SAEs encode each token independently and do not expose information that persists across a sequence.