arXiv Machine Learning

Why AI Detection Fails for Academic Integrity

arXiv:2608. 11256v1 Announce Type: new Abstract: Institutions use commercial AI detectors for academic integrity, yet detectors cannot distinguish AI editing from full LLM drafts and may treat both as misconduct.

arXiv AI
Aug 28

Style as a Confound: False Positives in AI Detection of Non-Native Academic Writing

The study examines how professional English editing influences AI text detectors’ false-positive rates for non-native academic writing. Using 135,389 pairs of original and edited manuscripts, researchers found that detector responses varied widely—some editors increased AI scores while others decreased them—and that score changes correlated with the extent of editing. These results highlight professional editing style as a key confounding factor in AI detection, complicating the distinction between AI authorship and linguistic style.

By Hyeonchu Park, Gahye Jeong, Bugeun Kim
arXiv AI
Jun 24

Policies Permitting LLM Use for Polishing Peer Reviews Are Currently Not Enforceable

arXiv:2603. 20450v2 Announce Type: replace-cross Abstract: A number of scientific conferences and journals have recently enacted policies that prohibit LLM usage by peer reviewers, except for polishing, paraphrasing, and grammar correction of otherwise human-written reviews.

By Rounak Saha, Gurusha Juneja, Dayita Chaudhuri, Naveeja Sajeevan, Nihar B Shah, Danish Pruthi
arXiv Machine Learning
Jun 5

Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection

arXiv:2606. 06481v1 Announce Type: cross Abstract: As AI writing assistants become increasingly integrated into real-world drafting and revision workflows, many documents are no longer purely human-written or AI-generated, but instead result from progressive human-AI co-editing.

By Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Tianjun Yao, Xinyi Shang, Yi Tang, Jiacheng Cui, Ahmed Elhagry, Salwa K. Al Khatib, Hao Li, Salman Khan, Zhiqiang Shen
arXiv AI
Sep 17

Does AI Assistance Leave a Temporal Fingerprint? Detecting Overreliance in AI-Assisted Writing and Programming

The study investigates whether AI assistance leaves a temporal fingerprint in writing and programming tasks. By analyzing keystroke-level data from three corpora, the authors find that AI contributions appear in distinct bursts and that temporal patterns can almost perfectly distinguish wholesale delegation from authentic work, though ordinary collaboration remains hard to detect. The research suggests that process visibility could serve as a basis for academic integrity checks.

By Eduardo Davalos, Yike Zhang
arXiv Machine Learning
Sep 14

Limits of LLM Text Detectors in Education

The paper "Limits of LLM Text Detectors in Education" argues that existing LLM‑generated text detectors assume a binary human/LLM distinction, which fails to capture realistic student‑AI collaboration. It introduces a contribution‑aware evaluation framework with eight student contribution levels and presents GEDE, a benchmark of over 900 human‑written and 12,500 generated essays across 886 tasks. Using GEDE, the authors evaluate four detection methods and find that most detectors perform poorly on intermediate contribution levels, especially LLM‑assisted revisions, raising concerns about false accusations.

By Lukas Gehring, Benjamin Paa{\ss}en
arXiv AI
Oct 9

Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing

The study examines how the rapid turnover of large language models (LLMs) undermines the effectiveness of AI‑detectors used to screen scientific manuscripts. By comparing 4,000 PNAS abstracts rewritten by 23 different LLM versions released between June 2023 and August 2026, the authors show that detectors trained on past versions can fail dramatically at generation boundaries, missing up to 96% of rewrites just after a sharp transition. Even detectors updated to newer versions can overlook rewrites of older models, and a commercial detector missed most rewrites of the newest version while flagging few human‑written abstracts.

By Kazuki Nakajima, Takayuki Mizuno
arXiv Computation and Language
5d ago

Likelihood Is Not Authorship: Can We Reliably Measure Human Contribution in AI-Assisted Writing?

The paper argues that current AI‑text detection methods, which estimate human contribution by measuring how predictive a user’s prompt is of the final text, can be easily manipulated. By introducing SPECIFICITY PADDING—adding random IDs to the prompt—the authors show that a title‑only request can appear far more human than one containing the article’s key ideas, and that scores can swing dramatically when the evaluator’s view of the prompt is altered. Across a range of large language models, the same documents can be scored as 85 % human or 15 % human depending solely on the prompt used, revealing a fundamental limitation of likelihood‑based metrics for reliably measuring human authorship. whyItMatters":"The study demonstrates that likelihood‑based scores cannot reliably establish human contribution without additional evidence of what the person added and how the text was produced."

By Hasan Amin, Zhuoran Lu, Ming Yin
arXiv AI
Oct 1

Policy-Conditioned AI-Use Detection: An Evidentiary Framework for Academic Publishing

The paper introduces a policy‑conditioned AI‑use detection framework for academic publishing, arguing that traditional AI detection tools misalign with venue rules by merely identifying AI‑generated text. Instead, the proposed system treats the governing policy as an explicit input, generating hypotheses, evidence, calibration, and uncertainty rather than binary verdicts. It outlines how to benchmark compliance, evaluate true positive rates at venue‑specified false positive thresholds, and emphasizes the need for structured disclosure, tool routing, and contestable findings.

By Jairo Diaz-Rodriguez, Mumin Jia