arXiv AI
Jul 14
What Context Does a Coding Agent Actually Need to Act?
arXiv:2607. 09691v1 Announce Type: cross Abstract: A modern coding agent can hold an entire repository in its context window.
Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation
arXiv:2608. 13337v1 Announce Type: new Abstract: Sparse autoencoders are meant to name the things a language model computes, and the usual way to check that a latent matters is to switch it off and see what changes.
Related stories
arXiv Machine Learning
Jun 15
Decompose Sparsely Where You Should, Absorb Densely Where You Should No
arXiv:2606. 14040v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition.
arXiv Machine Learning
Jun 18
From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability
arXiv:2606. 18383v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable features from language models (LMs), yet a central question remains: when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM We study this through a post-hoc generalization framework that certifies the LM via a sparse proxy, obtained by replacing a native hidden activation with its pretrained SAE reconstruction.
arXiv Machine Learning
Aug 6
When More Becomes Less: Position-Dependent Repetition Effects in Language Models
arXiv:2608. 04021v1 Announce Type: cross Abstract: Cloze-style probes that vary how often a target token appears implicitly assume that more copies of a target affect prediction the same way regardless of where the readout slot sits.
Hugging Face Trending Papers
Aug 11
Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents
When a tool-using agent is given the same task in a different language, does it still take the same steps? Multilingual evaluation rarely asks: it compares final answers and discards the actions.
arXiv Machine Learning
Jul 15
From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness
arXiv:2607. 12166v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are the standard for decomposing superposed neural representations into interpretable features, and evaluation relies predominantly on correlational recovery metrics -- cosine similarity between ground-truth directions and decoder atoms.
arXiv AI
Jun 9
Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery
arXiv:2606. 09672v1 Announce Type: new Abstract: Ask a pretrained biomedical language model whether "cortisol 28 ug/dL" and "stock-market volatility" are related, and it returns a cosine similarity of 0.
arXiv AI
Jun 30
C$^{2}$R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders
arXiv:2606. 30609v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) are widely used to interpret large language models by decomposing activations into sparse, human-understandable features, but scaling to large dictionaries exposes fundamental challenges.
Hugging Face Trending Papers
Jun 8
Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery
Ask a pretrained biomedical language model whether "cortisol 28 ug/dL" and "stock-market volatility" are related, and it returns a cosine similarity of 0. 83 on a scale where 1.
arXiv Machine Learning
Jul 23
Building Fast, Evaluating Slow: Pipeline Choices Dominate Autointerpretability Score Variance
arXiv:2607. 19386v1 Announce Type: new Abstract: Cross-paper comparison of sparse autoencoder (SAE) interpretability often relies on autointerpretability scores.
arXiv Machine Learning
Jul 24
Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
arXiv:2607. 20596v1 Announce Type: new Abstract: Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested.
arXiv Machine Learning
Aug 6
Kathleen Writes: Autoregressive Generation and Data Scaling Without Attention
arXiv:2608. 04678v1 Announce Type: cross Abstract: Papers 1-2 of the Kathleen series showed that a byte-level, attention-free architecture built from a wavetable encoder and multi-scale reverberant state can match strong baselines on classification at ~450-700K parameters, without pretraining.