← Back to all news
arXiv Computation and Language By Zhibo Zhang, Yuxi Li, Zhen Ouyang, Ling Shi, Kailong Wang

RASET: Router-Agnostic Safety-Critical Expert Tuning Exposes Localized Safety Enforcement Failures in Mixture-of-Experts LLMs

Read the original on arXiv Computation and Language →

The paper introduces RASET, a router‑agnostic safety‑critical expert tuning framework for Mixture‑of‑Experts (MoE) large language models. RASET identifies a small subset of experts that are responsible for safety enforcement and applies parameter‑efficient tuning only to those experts, preserving the model’s intrinsic routing behavior. Experiments on five open‑weight MoE backbones show that RASET achieves a high safety‑bypass yield, outperforming existing baselines by a significant margin.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Oct 8

Routing-Aware Safety Alignment for Mixture-of-Experts Models

The paper introduces RASA, a routing‑aware safety alignment framework for Mixture‑of‑Experts (MoE) language models. RASA repairs Safety‑Critical Experts by selectively fine‑tuning only those experts that are disproportionately activated during jailbreak attacks, while keeping routing fixed and enforcing routing consistency with safety‑aligned contexts. Experiments on two MoE architectures show that RASA achieves near‑perfect robustness against diverse jailbreak attacks, generalizes across attacks, reduces over‑refusal, and maintains performance on standard benchmarks such as MMLU, GSM8K, and TruthfulQA.

By Jiacheng Liang, Yuhui Wang, Tanqiu Jiang, Ting Wang
llmsfine-tuningbenchmarkssafety
More like this →
arXiv AI
Jun 2

MESA: Improving MoE Safety Alignment via Decentralized Expertise

arXiv:2606. 00651v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures scale Large Language Models (LLMs) efficiently, enabling greater capacity with reduced computational cost by dynamically routing inputs to relevant experts, yet introduce a critical vulnerability: Safety Sparsity, where safety capabilities concentrate in few experts, making them susceptible to adversarial bypassing.

By Yitong Sun, Yao Huang, Teng Li, Ranjie Duan, Yichi Zhang, Xingjun Ma, Hui Xue, Xingxing Wei
llmsefficiencybenchmarkssafety
More like this →
arXiv Machine Learning
Aug 27

GRIP: Algorithm-Agnostic Machine Unlearning for Mixture-of-Experts via Geometric Router Constraints

The paper introduces GRIP, an algorithm‑agnostic framework for machine unlearning in Mixture‑of‑Experts large language models. GRIP enforces hard geometric constraints on router updates, projecting gradient changes into the null space of the retain set’s routing matrix to prevent routing manipulation. Two variants—training‑time stochastic projection and post‑training analytical correction—show significant improvements in routing stability, retain accuracy, and resistance to white‑box adversarial recovery across two MoE models.

By Andy Zhu, Rongzhe Wei, Yupu Gu, Pan Li
llmsroboticsbenchmarkssafety
More like this →