The paper introduces the Controlled Attribution Benchmark (CATT), a synthetic evaluation framework that fine‑tunes language models on entirely fictitious knowledge to provide clear ground truth for data attribution. By systematically varying single dataset properties—such as document redundancy and lexical distractors—CATT reveals that these factors consistently degrade attribution performance. The study also explores two underexamined design choices: using negative log‑odds gradients instead of cross‑entropy gradients, and aggregating only the top token attribution scores, which together improve ground‑truth recall by 35–72 percentage points and enhance counterfactual retraining performance on FTRACE‑TREX by up to 4.9×.
arXiv:2607. 23804v1 Announce Type: cross Abstract: Context attribution methods for large language models (LLMs) identify which input context contributes to the model response.
The paper introduces a method for identifying the dominant knowledge source behind large language model (LLM) outputs, distinguishing between faithfulness violations (misuse of provided context) and factuality violations (errors in internal knowledge). A simple linear probe trained on hidden representations can reliably classify this source, and the authors present AttriWiki, a self‑supervised pipeline that generates labeled training data by prompting models to recall withheld entities or read them from context. Probes trained on AttriWiki achieve high Macro‑F1 scores across several models and datasets, generalize zero‑shot to a benchmark, and show that attribution mismatches can increase error rates by up to 70%. "whyItMatters":"The study demonstrates that knowing the source of an LLM’s answer is crucial for effective mitigation of hallucinations, as attribution mismatches significantly raise error rates."
arXiv:2606. 04928v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed across diverse applications, raising critical questions for governance, accountability, and data provenance.
The paper introduces REASONS, a benchmark comprising 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution by large language models. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to assess the trade-off between reliability and responsiveness. Experiments on proprietary and open-source LLMs under various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but may increase abstention, while retrieval-augmented variants often maintain near-zero abstention. Human evaluation reveals a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.
arXiv:2607. 01420v1 Announce Type: cross Abstract: As grounded QA systems are increasingly deployed in AI assistants, accurately attributing generated answers to evidence is critical for user trust and model safety.
The paper introduces REASONS, a benchmark of 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution under different evidence conditions. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to balance reliability and responsiveness. Experiments with proprietary and open-source LLMs across various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but increases abstention, while adversarial metadata can push hallucination rates above 85%. Human evaluation confirms a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.
The paper introduces ATLAS, a likelihood‑based method for model authorship attribution (MAA) that learns how each source family generates text and attributes new text by comparing token‑level likelihoods. It also presents MAABench, a benchmark for evaluating MAA under out‑of‑distribution scenarios such as unseen domains and future model versions. Experiments show ATLAS outperforms existing baselines, achieving 74.27% accuracy on the hardest MAABench setting and correctly identifying teacher models in distillation experiments.
arXiv:2506.17585v4 Announce Type: replace Abstract: Large language models can answer questions correctly and still hallucinate their citations. Most citation systems address this by retrieving docume...
The paper introduces Causal Attribution Pruning (CAP), a training‑free technique that identifies critical attention heads in large language models by measuring their causal impact on reasoning tasks. CAP estimates performance loss when each head is masked on a small calibration set, then translates these causal scores into weight‑level importance for pruning. Compared to magnitude‑only or activation‑based methods, CAP achieves up to 61% relative accuracy gains over Wanda on ARC‑Challenge at 20% sparsity and improves performance on GSM8K, StrategyQA, and ARC‑Challenge across Llama‑3‑8B‑Instruct and Mistral‑7B‑Instruct at 10%, 20%, and 50% sparsity.
arXiv:2610.36755v1 Announce Type: cross Abstract: Gradient-based data attribution has become an important tool for high-quality data selection in the instruction fine-tuning of large language models...
arXiv:2608. 03859v1 Announce Type: cross Abstract: Large language models (LLMs) pose challenges to academic integrity and peer review.
MUDDLE is a benchmark designed to disentangle the effects of document length and topical distractors on document question‑answering systems. It contains 270 human‑annotated questions, each tested in five conditions: the source alone, the source with two or four hard negatives (topically similar), and the source with two or four random distractors matched in length and provenance. Experiments with GPT‑5‑mini show that hard negatives reduce accuracy more than length‑matched random distractors, indicating that topical similarity is a more significant source of error than length alone.