arXiv AI
Jun 16
Cordyceps: Covert Control Attacks on LLMs via Data Poisoning
arXiv:2605. 26595v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison.
Let Them Steal: Trapping Large Language Model Extraction Attacks with Knowledge Honeypot
arXiv:2606. 15810v1 Announce Type: cross Abstract: Large language models deployed as commercial APIs are vulnerable to model extraction attacks, while existing defenses either act too late or degrade utility for legitimate users.
Related stories
arXiv AI
Jun 9
Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions
arXiv:2604. 08304v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) extends large language models (LLMs) with external knowledge, but this access path also introduces security risks that existing work often conflates with inherent LLM flaws.
arXiv AI
Aug 6
Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
arXiv:2608. 04519v1 Announce Type: new Abstract: Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not.
arXiv AI
Jun 3
Inference Cost Attacks for Retrieval-Augmented Large Language Models
arXiv:2606. 02643v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG)-enhanced LLM systems, while powerful, introduce substantial inference costs due to the inclusion of an extra multi-stage pipeline that dynamically retrieves and synthesizes information from external knowledge sources.
Hugging Face Trending Papers
Aug 5
Leak-Resistant Unlearning: A New Benchmark for Evaluating Multi-Hop Reasoning Consistency and Recovery Robustness
Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks include mainly single-hop questions and a narrow set of multi-hop questions.
arXiv Machine Learning
Jun 4
Bypassing Prompt Guards in Production with Controlled-Release Prompting
arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.
arXiv AI
Aug 11
Stealing Reasoning Traces from Proprietary LLM APIs
arXiv:2608. 09867v1 Announce Type: cross Abstract: Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage.
arXiv Machine Learning
Jun 17
Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs
arXiv:2606. 17110v1 Announce Type: cross Abstract: Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets.
arXiv Machine Learning
Aug 4
Mitigating Backdoors via Decoy Shortcuts and Knowledge Decoupling
arXiv:2608. 00732v1 Announce Type: new Abstract: Backdoor attacks pose a serious threat to deep neural networks, especially when training relies on third-party data, allowing adversaries to inject malicious behaviors through data poisoning.
Hugging Face Trending Papers
Jul 29
ToxScreen: Detecting Whether an LLM Has Been Poisoned
As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned.
arXiv Machine Learning
Jul 30
ToxScreen: Detecting Whether an LLM Has Been Poisoned
arXiv:2607. 26849v1 Announce Type: cross Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time.
arXiv Machine Learning
Jun 5
Steering Vectors are an Adversarial Attack Surface
arXiv:2606. 05958v1 Announce Type: new Abstract: Activation steering has become a popular way to control Large Language Model (LLM) behavior without fine-tuning.