arXiv Machine Learning
Aug 27

Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity

Robust CurveMoE is a mixture‑of‑experts framework that protects neural networks against perturbations defined by multiple norm constraints. It connects norm‑specialized models through a low‑loss path, selectively expertises only influential layers, and shares the rest of the parameters across routing paths. The method introduces contribution‑guided partial updating to reduce curve‑construction cost and provides a theoretical bound on the objective gap between partial and full optimization, achieving consistent improvements in clean, norm‑specific, and Union accuracy on CIFAR‑100 and ImageNet‑100.

By Xu Zhang, Ren Wang
arXiv Machine Learning
Sep 17

SAM-on-the-Curve: Sharpness-Aware Mode Connectivity for Robust Weight-Space Interpolation

The paper introduces Sharp Mode Connectivity (SMC), a new formulation of mode connectivity that optimizes for flatness in the entire local neighborhood of a weight-space path rather than just along the path itself. By applying a sharpness-aware approximation to a minimax objective, SMC produces low-loss curves that remain robust to distribution shifts, achieving significant accuracy gains on CIFAR‑10‑C and outperforming standard mode connectivity on several architectures.

By Alejandro Calatrava, Xu Zhang, Ren Wang