arXiv AI August 28, 2026 By Greg Kocher, Robert West, Cl\'ement Dumas, Julian Minder

Diff Mining: Logit Differences Reveal Finetuning Objectives

Diff Mining is a framework that identifies what a finetuned language model has learned by comparing its logits to those of its base model. It extracts per-context logit differences on a reference corpus and aggregates them into an interpretable token set using either a Top‑K frequency method or Non‑negative Matrix Factorization. The approach outperforms existing model‑diffing methods in domain detection and bias identification, and it requires only access to output logits, making it scalable to large models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 2

Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

arXiv:2607. 01208v1 Announce Type: cross Abstract: Language models deployed in high-stakes roles can potentially favor certain entities, brands, or viewpoints, steering user decisions at scale.

By Shayan Talaei, Abhinav Chinta, Devvrit Khatri, Amin Karbasi, Azalia Mirhoseini, Amin Saberi

llms efficiency safety

arXiv AI

Jun 30

Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives

arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.

By Mohammed Abu Baker, Luca Baroni, Dan Wilhelm

llms agents fine-tuning benchmarks

arXiv AI

Jul 28

Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation

arXiv:2607. 22766v1 Announce Type: cross Abstract: The alignment of Large Language Models (LLMs) is increasingly bottlenecked by data quality.

By Yunting Song, Matthew Watson, Peter Grabowski, Jun Qin

llms reinforcement-learning benchmarks safety data-engineering

arXiv AI

Jun 8

Auditing Training Data in Domain-adapted LLMs: LoRA-MINT

arXiv:2606. 06946v1 Announce Type: cross Abstract: We present LoRA-MINT, a new methodology for Membership Inference Test (MINT) applied to recent Large Language Models (LLMs) fine-tuned for specific Natural Language Processing (NLP) tasks through Low-Rank Adaptation (LoRA).

By Gonzalo Mancera, Daniel DeAlcala, Aythami Morales, Julian Fierrez, Ruben Tolosana, Francisco Jurado

llms nlp fine-tuning benchmarks

arXiv Machine Learning

Sep 22

DiFA: Dual Evidence Fusion and Aggregation for Token-Level Text Anomaly Detection

arXiv:2609.22136v1 Announce Type: cross Abstract: Text anomaly detection, the task of identifying text instances that deviate from normal language patterns, is crucial for language-driven application...

By Yanyu Qian, Pengcheng Weng, Yue Tan, Enguang Zuo, Yu Zheng, Yixin Liu

benchmarks safety

arXiv AI

Jun 17

Combating Data Laundering in LLM Training

arXiv:2604. 01904v3 Announce Type: replace-cross Abstract: Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.

By Muxing Li, Zesheng Ye, Sharon Li, Feng Liu

llms benchmarks

One email a morning, machine-written

Related stories

Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation

Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives

Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation

Auditing Training Data in Domain-adapted LLMs: LoRA-MINT

DiFA: Dual Evidence Fusion and Aggregation for Token-Level Text Anomaly Detection

Combating Data Laundering in LLM Training