arXiv Machine Learning June 10, 2026 By Leonard Engmann, Christian Medeiros Adriano, Holger Giese

From Observation to Intervention: A Causal Audit of Expert Importance in Mixture-of-Experts Models

Read the original on arXiv Machine Learning →

arXiv:2606. 10703v1 Announce Type: new Abstract: Interpretability methods routinely use population-level summary statistics over observed model behaviour to license claims about the effects of targeted interventions on specific computations; in Pearl's terms, they treat rung-1 associational evidence as if it supported rung-2 interventional conclusions, a move whose validity is rarely tested.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 10

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

arXiv:2607. 08349v1 Announce Type: new Abstract: Mechanistic interpretability often evaluates explanations by intervening on a model: swapping hidden states, patching activations, ablating components, or comparing a compressed model to the original one.

By Amir Asiaee

llms safety

arXiv Machine Learning

Jul 31

Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

arXiv:2607. 28308v1 Announce Type: new Abstract: Sparse mixture-of-experts (MoE) language models route each token to multiple experts, suggesting a geometric account of their benefit: co-selected experts should contribute distinct representation directions.

By Huiyuan Tian, Bonan Xu, Shijian Li

llms efficiency

Hugging Face Trending Papers

Jul 30

Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

Sparse mixture-of-experts (MoE) language models route each token to multiple experts, suggesting a geometric account of their benefit: co-selected experts should contribute distinct representation directions. Existing evidence often conflates route coherence, candidate quality, and candidate-by-context interaction.

llms efficiency

arXiv Machine Learning

Jun 25

How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly Dissolves

arXiv:2606. 25092v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) models route each token to a few of many experts, inviting the hypothesis that experts form functional modules tied to capabilities or languages.

By Tony Salomone, Deep Gandhi, Ali Asaria

efficiency benchmarks

arXiv Machine Learning

Jun 29

The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching

arXiv:2606. 27510v1 Announce Type: new Abstract: Activation patching is the primary tool in mechanistic interpretability.

By Sankaran Vaidyanathan, David Arbour, Aaron Mueller, Scott Niekum, David Jensen

llms safety

arXiv Machine Learning

Jun 3

Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models

arXiv:2606. 03780v1 Announce Type: cross Abstract: Causal tracing of factual recall has been studied predominantly in dense transformer language models, where interventions localize information flow to layers or feed-forward modules.

By Yuetian Lu, Ali Modarressi, Yihong Liu, Hinrich Sch\"utze

llms rag

Related stories

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly Dissolves

The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching

Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models