The paper investigates how safety‑aligned large language models (LLMs) can still be tricked by jailbreaks or over‑refuse benign requests. By analyzing residual‑stream updates in decoder‑only Transformers, the authors find that harmfulness is more readily decoded from attention updates—especially in early layers—while the MLP component is primarily responsible for writing refusal responses. Post‑training enhances MLP‑driven refusal writing without improving harmfulness decodability, and causal interventions confirm that downstream MLP updates are crucial for the refusal shift caused by upstream attention changes. Building on these insights, the authors propose Sentinel, a system that combines an attention‑derived perception score with an MLP‑derived decision score to better detect jailbreaks and over‑refusals across multiple models.
By Zixuan Weng, Haoqi He, Weifei Jin, Yue Huang, Yang Bai
arXiv:2606. 07963v1 Announce Type: new Abstract: Backdoor attacks in large language models (LLMs) are often treated as isolated trigger-response failures, motivating defenses tailored to specific triggers or behaviors.
By Omar Mahmoud, Aly M. Kassem, Thommen George Karimpanal, Buddhika Laknath Semage, Negar Rostamzadeh, Golnoosh Farnadi, Santu Rana
arXiv:2608. 03201v1 Announce Type: new Abstract: Safety guards are widely used to filter harmful content and are typically trained via supervised fine-tuning on labeled prompt-response pairs.
By Yu Feng, Chunting Zang, Chen Shen, Rui Miao, Ge Teng, Weidong Cai, Jieping Ye
arXiv:2510.17021v2 Announce Type: replace-cross
Abstract: Large language model (LLM) unlearning is a key approach for removing undesired data, knowledge, or behaviors from pretrained models while ret...
By Bingqi Shang, Yiwei Chen, Yihua Zhang, Bingquan Shen, Sijia Liu
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.
By Deepanshu Mody, Samarth Agarwal, Utkarsh Mittal, Dipesh Mahato
arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).
By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao
arXiv:2508. 10029v3 Announce Type: replace-cross Abstract: Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations.
By Wenpeng Xing, Bohan Yang, Mohan Li, Chunqiang Hu, Haitao Xu, Ningyu Zhang, Bo Lin, Meng Han
arXiv:2606. 18322v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) decompose residual-stream activations into interpretable features.
By Mingyue Cui, Linghui Shen, Xingyi Yang
arXiv:2609.36570v1 Announce Type: cross
Abstract: Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that...
By Mark Russinovich
The paper investigates how trigger-based backdoors function in large language models by using sparse autoencoders (SAEs) to identify feature directions across layers and transformer components. In a controlled experiment, 1B and 8B models were made to continue English prompts in French or German when presented with fixed trigger sequences. The study finds that different SAE feature directions correspond to trigger detection, residual-stream propagation, and language tracking, with residual-stream features being most effective for controlling the backdoor behavior.
By Wissam Antoun, Francis Kulumba, Th\'eo Lasnier, Beno\^it Sagot, Djam\'e Seddah
arXiv:2607. 19894v1 Announce Type: cross Abstract: Large language models (LLMs) are vulnerable to backdoor attacks, where hidden triggers induce malicious outputs.
By Yuxi Li, Zhibo Zhang, Kailong Wang, Xingshuo Han, Ling Shi, Haoyu Wang
The paper introduces NEEDLE, a training‑free technique for removing backdoors from large language models. After a trigger is identified, NEEDLE estimates a backdoor direction and a refusal subspace using activation vectors, then applies sequential weight orthogonalisation to suppress the backdoor while preserving refusal‑related representations. The method requires no clean reference model or original poisoned data and achieves the lowest attack success rate and minimal impact on model performance across multiple model families and attack types.
By Minoo Kim, Vasileios Lampos, George Drayson