Understanding and Mitigating Role Spoofing in LLMs

The paper investigates how large language models (LLMs) can be tricked into attributing text to the wrong role, a phenomenon called role spoofing. It shows that LLMs rely on both content cues (semantics and style) and structural cues (control tokens, formatting) to track who said what, making them vulnerable to attacks that combine fake structure and content. The authors demonstrate that adversarial training can mitigate content-based attacks but not structure-based ones, and propose a linear-probe method to detect role spoofing across settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• safety