arXiv Machine Learning September 2, 2026 By Maryam Alshehyari, Dushyant Singh Chauhan, Samuele Poppi, Martin Takac, Salem Lahlou, Nils Lukas

CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMs

Read the original on arXiv Machine Learning →

CopyShield is a benchmark that compares three copyright‑defense methods—contrastive decoding, Direct Preference Optimization (DPO), and activation intervention—across two large language models (LLaMA‑3.1‑8B and Mistral‑7B‑v0.3). The study uses controlled memorization of five public‑domain books to measure literal leakage, calibrated non‑literal leakage, utility, and degeneracy, finding that each intervention level yields distinct compliance‑utility trade‑offs. Results show contrastive decoding limits degeneracy but hits a suppression floor, DPO nearly eliminates literal leakage yet causes paraphrase‑loop degeneracy, and activation intervention blocks most non‑literal queries before generation, with human evaluation highlighting coherence and perceived copyright risk differences.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language

3d ago

Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model

arXiv:2602.07120v3 Announce Type: replace Abstract: Language models (LMs) tend to memorize portions of their training data and emit verbatim spans. When the underlying sources are sensitive or copyri...

By Jacqueline He, Jonathan Hayase, Wen-tau Yih, Sewoong Oh, Luke Zettlemoyer, Pang Wei Koh

llms

arXiv AI

Jun 6

LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

arXiv:2606. 06286v1 Announce Type: cross Abstract: Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use.

By Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech

llms safety

arXiv AI

Jun 6

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

arXiv:2606. 05644v1 Announce Type: new Abstract: When retrieved evidence contradicts parametric memory, language models frequently ignore context and default to memorized priors -- a failure that undermines the core purpose of retrieval augmentation.

By Zhe Yu, Wenpeng Xing, Tiancheng Zhao, Mohan Li, Changting Lin, Meng Han

llms rag benchmarks safety

arXiv AI

Aug 3

TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Text

arXiv:2607. 28862v1 Announce Type: cross Abstract: The rapid development of Large Language Models (LLMs) has led to significant advances across a wide range of language tasks, while simultaneously raising growing concerns about unauthorized data exploitation and privacy leakage.

By Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu

llms nlp reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Aug 14

A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models

arXiv:2602. 18733v2 Announce Type: replace Abstract: Training data leakage from Large Language Models (LLMs) raises serious concerns related to privacy, security, and copyright compliance.

By Trishita Tiwari, Ari Trachtenberg, G. Edward Suh

llms safety

arXiv AI

Jun 3

Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing

arXiv:2606. 02822v1 Announce Type: cross Abstract: Production LLM applications stack several defense families -- refusal-phrase filters, token-budget controls, model allowlists, rate limits, tool-registry authentication -- yet existing breach-and-attack-simulation (BAS) benchmarks report a single aggregate coverage number, hiding which family closes which threat.

By Alexandre Cristov\~ao Maiorano

llms agents benchmarks safety

One email a morning, machine-written

Related stories

Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model

LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Text

A Prior-Aware Metric for Efficiently Distinguishing Memorization from Generalization in Large Language Models

Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing