← Back to all news
arXiv Machine Learning By Maryam Alshehyari, Dushyant Singh Chauhan, Samuele Poppi, Martin Takac, Salem Lahlou, Nils Lukas

CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMs

Read the original on arXiv Machine Learning →

CopyShield is a benchmark that compares three copyright‑defense methods—contrastive decoding, Direct Preference Optimization (DPO), and activation intervention—across two large language models (LLaMA‑3.1‑8B and Mistral‑7B‑v0.3). The study uses controlled memorization of five public‑domain books to measure literal leakage, calibrated non‑literal leakage, utility, and degeneracy, finding that each intervention level yields distinct compliance‑utility trade‑offs. Results show contrastive decoding limits degeneracy but hits a suppression floor, DPO nearly eliminates literal leakage yet causes paraphrase‑loop degeneracy, and activation intervention blocks most non‑literal queries before generation, with human evaluation highlighting coherence and perceived copyright risk differences.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 10

Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

The paper demonstrates that fine‑tuning large language models on a single author’s works can trigger the models to reproduce large verbatim excerpts from copyrighted books, even when prompted only with semantic descriptions. Experiments on GPT‑4o, Gemini‑2.5‑Pro, and DeepSeek‑V3.1 show up to 85‑90% recall of held‑out books, with spans exceeding 460 words, and this effect generalizes across authors and model providers. The findings suggest that fine‑tuning reactivates latent memorization from pre‑training, revealing a widespread vulnerability in industry models.

By Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg, Tuhin Chakrabarty
llmsreinforcement-learningfine-tuningsafety
More like this →
arXiv AI
2d ago

LLMs Can Leak Training Data But Do They Want To? Evaluating Memorization Propensity in LLMs

The paper introduces PropMe, a framework that evaluates both the capability of large language models (LLMs) to reproduce training data via prefix attacks and their natural propensity to do so without adversarial prompting. Using SimpleTrace to compare every generation against the full indexed training corpus, the authors test models such as Comma, DFM Decoder, and OLMo3 32B across datasets like Common Pile, Dynaword, and Dolma3. Results show that natural sentence prompts produce less training text than prefix attacks, but unprompted or single-word prompts can reproduce as much or more, often involving duplicated licenses, code, and templates; continual pre‑training only modestly reduces verbatim reproduction.

By Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech
llmssafety
More like this →
arXiv AI
2d ago

TextCloak: Safeguarding Data from Unauthorized LLM Fine-Tuning via RL-Driven Unlearnable Examples

TextCloak is a reinforcement‑learning driven framework that generates unlearnable examples (UEs) to protect textual data from unauthorized fine‑tuning of large language models (LLMs). It transforms clean text into perturbations that preserve meaning and naturalness while degrading the utility of models trained on them. Experiments on six datasets and nine LLMs show that TextCloak consistently impairs unauthorized fine‑tuning and remains robust across different architectures, training settings, and adaptive attacks.

By Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu
llmsreinforcement-learningfine-tuningbenchmarkssafety
More like this →