arXiv Machine Learning

Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability

arXiv:2509. 23689v2 Announce Type: replace Abstract: Model Merging (MM) has proven to be an effective alternative to multi-task learning, where several fine-tuned models are merged, without access to the tasks' training data, into one model that retains performance across different tasks.

arXiv Machine Learning
Aug 27

Rethinking the Transferable Adversarial Attacks and Robust Defense in Federated Learning

The paper investigates how adversarial examples transfer between client models in federated learning and explores the relationship between these examples and client data distributions. It proposes a defense strategy based on adversarial training that leverages the transferability of model robustness. Experiments on real-life datasets demonstrate that the new attack and defense methods outperform existing state‑of‑the‑art approaches.

By Zuobin Xiong, Deval Mukherjee, Homook Cho, Wei Li
arXiv AI
3d ago

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

The paper introduces FAB, an attack that uses meta‑learning to embed dormant adversarial behaviors into large language models (LLMs). These behaviors remain inactive until the model is finetuned by downstream users, at which point the model can exhibit unwanted actions such as unsolicited advertising, jailbreakability, or over‑refusal. FAB is shown to be effective across multiple LLMs and resilient to various finetuning settings.

By Thibaud Gloaguen, Mark Vero, Robin Staab, Martin Vechev
Hugging Face Trending Papers
Jun 2

RogueMerge: Robust and Unified Attacks against LLM Model Merging

Model merging composes specialized capabilities into a single LLM by aggregating task vectors sourced from unverified public platforms, exposing a critical supply-chain attack surface: Because any malicious behavior can be encoded into a task vector, and merging grants third-party vectors direct write access to model weights, an attacker-provided task vector can enable or amplify diverse downstream threats. Prior work studies only backdoor attacks against model merging for classifiers using static arithmetic heuristics, which fail to effectively handle diverse attacks on generative LLMs for three reasons.