arXiv AI
Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 25
Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning
Mechanistic Localization bridges mechanistic interpretability and post-training optimization by isolating critical parameters via interpretative approaches and then guiding parameter-efficient Supervi...
arXiv Machine Learning
Jun 9
PriFT: Prior-Support Guided Supervised Fine-Tuning
arXiv:2606. 09396v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation and often serves as the initialization stage for reinforcement learning (RL), but it can show weaker generalization than RL.
arXiv Machine Learning
Jul 2
Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol
arXiv:2607. 00089v1 Announce Type: new Abstract: Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact.
arXiv AI
Sep 2
S^3martCirc: Self-supervised Smart Circuit Discovery
S^3martCirc is a self‑supervised framework that jointly discovers and interprets neural circuits in large language models, rather than treating circuit discovery and functional interpretation as separate stages. It abstracts node behavior into two general computational roles that generalize across tasks and introduces a quantitative metric for assigning these roles, enabling simultaneous identification of importance and function. Experiments demonstrate that S^3martCirc outperforms existing methods in circuit discovery.
arXiv Machine Learning
Jul 1
A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning
arXiv:2602. 20062v2 Announce Type: replace Abstract: Pretraining and fine-tuning are central stages in modern machine learning systems.
arXiv AI
Sep 10
SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
The paper introduces SAEScientist-Bench, a benchmark that tests whether AI agents can autonomously conduct mechanistic interpretability research using Sparse Autoencoders (SAEs). Agents are tasked with designing contrastive probes and navigating a large feature dictionary in Gemma-2-9B-IT to identify optimal features for a target concept, with performance measured against expert-curated references on activation rank, concept selectivity, and causal steering. Results show that while frontier agents can discover features and outperform controls, they still lag behind expert baselines, especially in causal steering, highlighting both the potential and current limitations of closed-loop autonomous AI research.