arXiv AI
Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 3
SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment
The paper introduces SEAL, a training-time, parameter‑efficient defense that attaches a plug‑and‑play adapter to the shared expert component of Mixture‑of‑Experts models, and SEAL++, which adds an orthogonal constraint to preserve existing safety subspaces. By leveraging the always‑activated shared expert, SEAL mitigates the structural vulnerability of sparse routing to adversarial manipulation, reducing attack success rates by up to 60% with minimal impact on model capability. The approach is evaluated across six attack scenarios involving harmful prompting, jailbreaks, malicious fine‑tuning, and neuron pruning.
arXiv AI
Aug 6
Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning
arXiv:2608. 05045v1 Announce Type: cross Abstract: Released aligned large language models remain vulnerable to malicious downstream finetuning.
arXiv Machine Learning
Sep 22
Decomposing Refusal Steering in Mixture-of-Experts Models
arXiv:2606.04160v2 Announce Type: replace-cross Abstract: Safety alignment in instruction-tuned large language models (LLMs) depends on a model's ability to reliably refuse harmful or disallowed requ...
arXiv Computation and Language
Aug 27
Groundhog Bit-Flip Attack: Seeding Infinite Generation Loops in Mixture-of-Experts LLMs through Bit Flips
The paper introduces the Groundhog Bit-Flip Attack (GBFA), a novel denial-of-service attack targeting Mixture-of-Experts (MoE) large language models (LLMs). By flipping specific routing-layer bits that activate certain experts, GBFA can cause models to generate excessively long outputs—up to a 5912% increase in token usage—while largely preserving semantic content. The attack requires deactivating fewer than four experts on average across four real-world MoE-based LLMs, exposing a significant robustness vulnerability in these architectures.
arXiv Computation and Language
Aug 24
RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models
The paper introduces RARE, a router‑agnostic representation engineering framework for Mixture‑of‑Experts language models. RARE projects behavioral perturbations onto the null space of the router matrix to avoid affecting routing, and corrects downstream routing drift. Experiments on six open‑weight MoE models show that RARE improves steering tasks—reducing harmfulness, increasing truthfulness, and enhancing factual editing—while preserving overall model accuracy.
arXiv Computation and Language
Aug 25
RASET: Router-Agnostic Safety-Critical Expert Tuning Exposes Localized Safety Enforcement Failures in Mixture-of-Experts LLMs
The paper introduces RASET, a router‑agnostic safety‑critical expert tuning framework for Mixture‑of‑Experts (MoE) large language models. RASET identifies a small subset of experts that are responsible for safety enforcement and applies parameter‑efficient tuning only to those experts, preserving the model’s intrinsic routing behavior. Experiments on five open‑weight MoE backbones show that RASET achieves a high safety‑bypass yield, outperforming existing baselines by a significant margin.