arXiv AI By Luman Zhao, Minghui Xu, Yue Zhang, Yijun Yang

LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense

Read the original on arXiv AI →

The paper introduces Learnable Trust‑Boundary Delimiters (LTBD), a lightweight defense that embeds learnable delimiters into LLM inputs to explicitly mark trusted user instructions versus untrusted external data. LTBD keeps the model’s parameters unchanged, achieving superior performance over inference‑time defenses and matching training‑based methods while adding negligible inference overhead. Experiments show near‑zero attack success rates on AlpacaFarm and TaskTracker, and the method remains robust even against adaptive attackers who know the defense.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 26

Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors

The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.

By Joshua Penman
arXiv Machine Learning
2d ago

LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation

The paper introduces LLM4Trust, a benchmark framework that evaluates large language models (LLMs) for trust assessment in cybersecurity. It constructs trust graphs to represent five fundamental trust properties, designs property‑understanding tasks, and tests eight LLMs across nine prompting strategies. The study identifies effective LLM‑prompt combinations, applies them to real datasets, proposes methods to handle context limits, and demonstrates that LLMs can understand trust properties while highlighting vulnerabilities and high inference costs, for which defense mechanisms and batch inference are suggested.

By Jie Wang, Yanbo Sun, Zheng Yan, Jiahe Lan, Elisa Bertino
Hugging Face Trending Papers
Jul 29

ToxScreen: Detecting Whether an LLM Has Been Poisoned

As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned.

arXiv Machine Learning
Jul 30

ToxScreen: Detecting Whether an LLM Has Been Poisoned

arXiv:2607. 26849v1 Announce Type: cross Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time.

By Anthony Hughes, Nicole Xing, Collin Francel, Andy Kim, Andrew Draganov