arXiv AI

Robust Bidirectional Associative Memory via Regularization Inspired by the Subspace Rotation Algorithm

arXiv:2511. 11902v2 Announce Type: replace-cross Abstract: Bidirectional Associative Memory (BAM) trained with Bidirectional Backpropagation (B-BP) often suffers from poor robustness and high sensitivity to noise and adversarial attacks.

arXiv AI
Sep 10

SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs

The paper introduces Dynamic DAE Guardrails (DSG), a method that uses Dynamic Sparse Autoencoders to perform precision unlearning in large language models. DSG leverages principled feature selection and a dynamic classifier to target activation-based unlearning, outperforming existing gradient‑based methods in terms of computational efficiency, stability, sequential unlearning, resistance to relearning attacks, data efficiency, and interpretability.

By Aashiq Muhamed, Jacopo Bonato, Mona Diab, Virginia Smith
arXiv AI
Jun 2

SORA: Free Second-Order Attacks in Fast Adversarial Training

arXiv:2606. 00738v1 Announce Type: cross Abstract: Adversarial Training (AT) is a leading defense against adversarial examples but often suffers from Catastrophic Overfitting (CO) in efficient single-step variants, where robustness to multi-step attacks collapses despite high single-step performance.

By Mazdak Teymourian, Ramtin Moslemi, Farzan Rahmani, Mohammad Hossein Rohban
arXiv AI
Oct 31

Sparse Fine-Tuning for Parameter-Efficient Adversarial Training

The paper introduces Sparse Adversarial Fine‑Tuning (SAFT), a method that identifies a small set of Robustness‑Critical Parameters (RCPs) through adversarial gradient saliency and updates only these parameters during adversarial training. By focusing on this sparse subspace, SAFT achieves higher clean and robust accuracy than LoRA‑based parameter‑efficient methods and matches or surpasses full‑parameter adversarial training while training only about 5% of the parameters. The approach is validated across multiple architectures and datasets, demonstrating its effectiveness in reducing memory cost without sacrificing robustness.

By Zhaoxin Wang, Zihang Ding, Handing Wang
arXiv Machine Learning
Aug 20

Gradient Mirage: Trainable yet Label-Unidentifiable Gradients in Large Language Model Split Learning

Gradient Mirage is a defense for large language model split learning that disrupts the assumption that exposed gradients faithfully reflect the client’s full-label training objective, thereby thwarting gradient matching attacks. It introduces inconsistencies across objective, direction, and scale by using selective autoregressive supervision, scale blinding, and directional privatization with a von Mises-Fisher mechanism. The method preserves utility through dual-track backpropagation and bottom-gradient recovery, achieving a stronger privacy‑utility trade‑off in experiments.

By Shiyu Miao, Yunlong Mao, Zirui Huang, Liang Yao, Tianshuo Zheng, Yanhui Gu, Fan Liu, Sheng Zhong
arXiv AI
Jun 18

Revealing Hidden Vulnerabilities in Autoencoders through Gradient Signal Restoration

arXiv:2505. 03646v5 Announce Type: replace-cross Abstract: Adversarial robustness of deep autoencoders (AEs) has received less attention than that of discriminative models, although their compressed latent representations induce ill-conditioned mappings that can amplify small input perturbations and destabilize reconstructions.

By Chethan Krishnamurthy Ramanaik, Arjun Roy, Tobias Callies, Eirini Ntoutsi