← Back to all news
arXiv Machine Learning By Alexandru-Iulius Jerpelea, Amith Ananthram

A Deeper Analysis of Block-Sparse Featurizers

Read the original on arXiv Machine Learning →

The paper investigates the block-sparse featurizer (BSF), a model that uses small subspaces as atomic units instead of single directions, aiming to capture features on low-dimensional manifolds common in vision. It identifies that BSF still exhibits classic sparse autoencoder failure modes such as feature splitting and composition. The authors propose architectural improvements, notably a Tournament Top‑K selection rule, which markedly reduces feature splitting, and they extend the block concept to a crosscoder framework.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 19

Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

The paper introduces MASS, a hierarchical data selection method that first groups data using low‑dimensional manifold coordinates learned by a dense autoencoder, then applies a TopK sparse autoencoder for quality‑aware feature coverage within each group. This approach addresses the shortcomings of traditional diversity metrics that mix semantic, supervisory, and noise signals. Experiments on Vision Flan and LLaVA‑CoT demonstrate that MASS outperforms existing baselines across various budgets and can match or exceed full‑data training with only a small subset.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu
llmsragfine-tuning
More like this →
arXiv Computer Vision
Aug 31

ShiftSplit-AD: Separating Domain Shift from Defects in Foundation-Feature Visual Anomaly Detection

ShiftSplit-AD is a method that separates domain shift from defects in visual anomaly detection by decomposing the residual matrix of DINOv2 features into low‑rank and row‑sparse components. The sparse component is used for scoring anomalies, optionally fused with the low‑rank part. Experiments on AeBAD‑S show that sparse‑only scoring raises image AUROC from 0.6780 to 0.7294 and AUPRC from 0.8052 to 0.8465, but it also lowers clean AUROC on MVTec categories and hurts Bottle localization, highlighting a trade‑off between filtering shift and preserving defect information.

By Muhamathu Ameer Ali Aacaas Muhamath
More like this →