arXiv:2607. 17425v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) compress model activations into sparse codes, but equal reconstruction error and sparsity can preserve different linearly decodable signals.
arXiv:2607. 05531v1 Announce Type: new Abstract: Variational Autoencoders (VAEs) frequently suffer from posterior collapse, a failure mode in which the approximate posterior converges to the prior, rendering the latent code uninformative.
The paper introduces a lightweight, observation-conditioned latent energy prior to improve inference for frozen implicit neural representation (INR) decoders when only sparse off‑grid signed distance function (SDF) samples are available. By standardizing latent codes based on a permutation‑invariant encoding of the sparse observations and combining this energy with a validation‑selected L2 prior, the method consistently outperforms baseline L2 and a six‑component Gaussian mixture model prior on both a controlled cell‑nucleus SDF dataset and a MedShapeNet‑derived SDF completion dataset, especially in the sparsest regimes. Ablation studies confirm that the energy term’s contribution is specific to the observed context rather than generic. whyItMatters":"The approach demonstrates that pretrained INR decoders can become more observation‑aware without retraining, improving shape completion accuracy in data‑sparse scenarios."
arXiv:2606. 14040v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition.
arXiv:2607. 01799v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) decompose internal activations of neural networks into sparse linear combinations of learned features by fitting an overcomplete dictionary $\mathbf{W}\in\mathbb{R}^{m\times n}$ with $m<n$, and inferring a sparse code $\mathbf{x}\in\mathbb{R}^n$ from $\mathbf{h}\approx\mathbf{W}\mathbf{x}$.
The paper introduces a training‑adaptive convolutional sparse coding (CSC) framework that learns the sparsity coefficient jointly with network parameters using an unfolded FISTA optimization. By treating the coefficient as a differentiable variable, the method balances information retention and compression through an information bottleneck perspective, promoting compact yet task‑relevant representations. A label‑free post‑training strategy further adjusts compression for corrupted inputs, yielding competitive accuracy on clean data and enhanced robustness to perturbations on CIFAR and ImageNet.
arXiv:2609.19122v1 Announce Type: new Abstract: Visual signals require compact yet sufficient representations for robust downstream prediction. Convolutional sparse coding (CSC) provides an explicit...
arXiv:2606. 14990v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are standard tools for mechanistic interpretability, but current SAE families are constrained by fixed encoder nonlinearities such as ReLU, JumpReLU, and TopK.
arXiv:2609.22950v1 Announce Type: cross Abstract: While compressed-sensing regularizers enable interpretable reconstruction of 4D Flow CMR through transparent variational objectives, their hand-craft...
arXiv:2606. 27321v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have become a leading tool for interpreting the representations of vision foundation models, decomposing their polysemantic activations into a larger set of sparse, more monosemantic features.
arXiv:2606. 18538v1 Announce Type: new Abstract: One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function.
arXiv:2606. 05389v1 Announce Type: new Abstract: Lossy compression is essential for massive spatiotemporal data from scientific simulations.
arXiv:2602. 14687v2 Announce Type: replace-cross Abstract: Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations.