The paper introduces Learnable Trust‑Boundary Delimiters (LTBD), a lightweight defense that embeds learnable delimiters into LLM inputs to explicitly mark trusted user instructions versus untrusted external data. LTBD keeps the model’s parameters unchanged, achieving superior performance over inference‑time defenses and matching training‑based methods while adding negligible inference overhead. Experiments show near‑zero attack success rates on AlpacaFarm and TaskTracker, and the method remains robust even against adaptive attackers who know the defense.
arXiv:2505.15738v3 Announce Type: replace-cross Abstract: Large language models are increasingly deployed in user-facing products that handle sensitive personal data while interacting with untrusted...
The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.
arXiv:2605. 26595v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison.
The paper introduces LLM4Trust, a benchmark framework that evaluates large language models (LLMs) for trust assessment in cybersecurity. It constructs trust graphs to represent five fundamental trust properties, designs property‑understanding tasks, and tests eight LLMs across nine prompting strategies. The study identifies effective LLM‑prompt combinations, applies them to real datasets, proposes methods to handle context limits, and demonstrates that LLMs can understand trust properties while highlighting vulnerabilities and high inference costs, for which defense mechanisms and batch inference are suggested.
As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned.
arXiv:2607. 26849v1 Announce Type: cross Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time.
arXiv:2607. 00481v1 Announce Type: cross Abstract: Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs).
The paper investigates how inference optimization for large language models can introduce numerical inconsistencies that trigger hidden backdoors. It introduces two types of optimization‑triggered backdoors: the Input‑Specific Optimization Backdoor (ISOB) and the Universal Optimization Backdoor (UOB), the latter enabling a model to remain benign under normal execution but activate a backdoor when optimization is applied. Experiments on seven open‑source LLMs, across multiple tasks and optimization backends, show UOB can achieve up to 100% attack success while maintaining clean accuracy, and the authors propose three defenses that reduce the attack success rate to 0.02.
The paper examines the vulnerability of open‑weight large language model safeguards that aim to block harmful usage by assuming new harmful behavior arises only through fine‑tuning. It demonstrates that simple, low‑cost attacks—abliteration and prefilling—can significantly increase attack success rates across multiple benchmarks, and proposes an abliteration‑resistant tuning (ART) method that mitigates these attacks. The study shows that safeguarding defenses must be evaluated against a broader range of attack strategies beyond fine‑tuning.
Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks, yet recent adaptive evaluations show that these results collapse once the attacker is allowed to optimize against the deployed defense.
arXiv:2606. 27567v1 Announce Type: cross Abstract: Prompt injection is the top security risk for LLM-integrated applications, yet every defense proposed so far has been broken.
arXiv:2608. 08027v1 Announce Type: cross Abstract: Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data.