arXiv Machine Learning

Who's Behind the Text? Robust Model Authorship Attribution

The paper introduces ATLAS, a likelihood‑based method for model authorship attribution (MAA) that learns how each source family generates text and attributes new text by comparing token‑level likelihoods. It also presents MAABench, a benchmark for evaluating MAA under out‑of‑distribution scenarios such as unseen domains and future model versions. Experiments show ATLAS outperforms existing baselines, achieving 74.27% accuracy on the hardest MAABench setting and correctly identifying teacher models in distillation experiments.

arXiv AI
Sep 10

Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

The paper demonstrates that fine‑tuning large language models on a single author’s works can trigger the models to reproduce large verbatim excerpts from copyrighted books, even when prompted only with semantic descriptions. Experiments on GPT‑4o, Gemini‑2.5‑Pro, and DeepSeek‑V3.1 show up to 85‑90% recall of held‑out books, with spans exceeding 460 words, and this effect generalizes across authors and model providers. The findings suggest that fine‑tuning reactivates latent memorization from pre‑training, revealing a widespread vulnerability in industry models.

By Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg, Tuhin Chakrabarty
arXiv AI
Jun 12

Authorship Attribution in Multilingual Machine-Generated Texts

arXiv:2508. 01656v2 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) have reached human-like fluency and coherence, distinguishing machine-generated text (MGT) from human-written content becomes increasingly difficult.

By Lucio La Cava, Dominik Macko, R\'obert M\'oro, Ivan Srba, Andrea Tagarelli
arXiv AI
Sep 10

Attribution in Scientific Literature: New Benchmark and Methods

The paper introduces REASONS, a benchmark of 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution under different evidence conditions. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to balance reliability and responsiveness. Experiments with proprietary and open-source LLMs across various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but increases abstention, while adversarial metadata can push hallucination rates above 85%. Human evaluation confirms a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.

By Deepa Tilwani, Yash Saxena, Seyedali Mohammadi, Ankur Padia, Edward Raff, Amit Sheth, Srinivasan Parthasarathy, Manas Gaur
arXiv Computation and Language
Oct 9

CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence

CiteVQA is a benchmark that evaluates multimodal large language models on both answering document questions and providing element‑level bounding‑box citations for the evidence used. It contains 1,897 questions across 711 PDFs in seven domains and two languages, with ground‑truth citations generated via an automated pipeline and strict quality control. The evaluation metric, Strict Attributed Accuracy (SAA), rewards a prediction only when both the answer and the cited region are correct, revealing widespread attribution hallucination in current models.

By Dongsheng Ma, Jiayu Li, Zhengren Wang, Yijie Wang, Jiahao Kong, Weijun Zeng, Jutao Xiao, Jie Yang, Bangrui Xu, Yuhan Wang, Bin Wang, Conghui He
arXiv AI
Sep 3

MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts

MultiGhostBench is a multilingual benchmark for authorship attribution of long-form text generated by large language models. It contains 928 books produced by five recent LLMs in six languages and three scripts, each averaging about 59,000 words, and is designed to test attribution methods under domain, author, and language shifts. Experiments show that no single attribution method dominates across all settings, with performance generally dropping under distribution shifts, and that transformer-based detectors retain generator information across languages while statistical and fingerprint-based detectors are more language‑dependent.

By Matteo Greco, Anudeex Shetty, Andrea Tagarelli, Jey Han Lau
Hugging Face Trending Papers
Sep 2

MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts

MultiGhostBench is a multilingual benchmark for authorship attribution of long‑form text generated by large language models. It contains 928 books produced by five recent LLMs in six languages and three scripts, each averaging about 59,000 words, and is designed to test attribution methods under domain, author, and language shifts. Experiments show that no single attribution method dominates across all settings, with performance generally dropping under distribution shifts, while transformer‑based detectors retain generator information across languages but vary in transfer effectiveness, and statistical/fingerprint detectors are more language‑dependent.