Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens
Read the original on arXiv AI →The paper introduces Sparse Readout Prism (SRP), a method that decomposes a language model’s readout matrix into sparse features, allowing logit‑lens scores to be expressed as sums of feature contributions. SRP reveals that lens readings depend on the corpus used to fit the readout, a phenomenon called corpus conditionality, and that the dominant readout feature remains stable across different corpora. By replacing the original readout with SRP’s sparse approximation, the authors recover 8.9–17.3 percentage points more of the tested logit differences than six geometric‑relation baselines, and ablating features shifts logit differences proportionally to their SRP contributions.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.