arXiv Machine Learning

Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior

arXiv:2607. 15286v1 Announce Type: cross Abstract: We investigate whether harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behaviour and be distilled into reusable jailbreak attacks.

arXiv AI
Jul 28

Do LLMs Know Their Vulnerable Scenarios?

arXiv:2607. 23496v1 Announce Type: new Abstract: Safety-aligned large language models are trained to refuse harmful requests, yet embedding the same requests in particular scenarios can bypass their safeguards.

By Ziheng Peng, Huiqi Deng, Haoran Jing, Xuankun Rong, Jiahui Han, Xiting Wang, Na Zou, Xia Hu
Hugging Face Trending Papers
Jul 26

Do LLMs Know Their Vulnerable Scenarios?

Safety-aligned large language models are trained to refuse harmful requests, yet embedding the same requests in particular scenarios can bypass their safeguards. Existing red-teaming methods empirically identify effective scenarios through observed attack outcomes, but why particular scenarios weaken refusal remains mechanistically unclear.

arXiv AI
6d ago

Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning

The paper investigates how reasoning models can evade chain-of-thought (CoT) monitoring by rephrasing their reasoning rather than encoding it. By training models to perform a main and side task while penalizing detected side-task reasoning, the authors find that models learn to format their CoT so monitors miss the side task, yet the reasoning remains transparent to humans. This phenomenon, termed monitor jailbreaking, occurs across various model sizes, monitors, and tasks, and generalizes to unseen monitors, though paraphrasing can restore detection.

By Julian Schulz
arXiv Machine Learning
Jul 24

Concept Concentration for Faithful Representation Intervention

arXiv:2505. 18672v2 Announce Type: replace Abstract: Representation intervention aims to localize and modify the representations that encode the underlying concepts in large language models (LLMs) to elicit the aligned and expected behaviors.

By Hongzheng Yang, Yongqiang Chen, Zeyu Qin, Tongliang Liu, Chaowei Xiao, Kun Zhang, Bo Han