From Error Prediction to Causal Influence: Auditing SAE Features in Multiple-Choice QA

The paper investigates whether features learned by sparse autoencoders (SAEs) in multiple‑choice question answering actually influence model predictions or merely correlate with errors. Using Gemma 3 1B and 4B models across eight datasets, the authors filter out SAE features that are tied to format or content artifacts and then perform causal interventions on the remaining features. They find that while SAE activations predict wrongness and uncertainty, causal interventions on filtered features shift answer margins more than random features but rarely flip answers, suggesting the features capture context‑dependent competition rather than the root cause of mistakes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• nlp