arXiv Computation and Language

A Systematic Analysis of Linguistic Features in AI-Generated English Text Detection Across Domains and Models

The paper presents a large‑scale empirical study of 284 interpretable linguistic features to detect AI‑generated English text across 30 language models and 10 text domains. It finds that classifiers using only linguistic features can reliably distinguish machine‑from human‑written text, but most indicators are highly context‑dependent. Only lexical richness measures consistently generalize across model families and domains, offering robust signals for detection.

arXiv AI
Oct 27

Robust and Fine-Grained Detection of AI Generated Texts

arXiv:2504.11952v4 Announce Type: replace-cross Abstract: An ideal detection system for machine generated content is supposed to work well on any generator as many more advanced LLMs come into existe...

By Ram Mohan Rao Kadiyala, Siddartha Pullakhandam, Kanwal Mehreen, Drishti Sharma, Siddhant Gupta, Jebish Purbey, Ashay Srivastava, Subhasya TippaReddy, Arvind Reddy Bobbili, Suraj Telugara Chandrashekhar, Ankan Ghosh, Modabbir Adeeb, Srinadh Vura, Suman Debnath, Hamza Farooq
arXiv Computation and Language
2d ago

Specificity Bias and the Wikipedia Paradox: An xAI Analysis of AI Text Detectors under Domain Shift

The paper examines how fine‑tuned Transformer detectors (DistilBERT, RoBERTa) perform on an out‑of‑distribution dataset of hyper‑local news generated by LLMs. Using SHAP‑based explainable AI, the authors uncover three phenomena: a Call‑to‑Action bias, a Specificity Bias where hallucinated named entities evade detection, and a Wikipedia Paradox where formal human prose is misclassified as AI. These results highlight systematic shortcomings of purely statistical detectors in structured human domains and underscore the need for semantically‑aware, interpretable detection methods.

By Nico Morloher, Johannes Reschke, Mohamed Mady
arXiv Computation and Language
Oct 19

Beyond Surface Cues: Robust LLM-Generated Korean Text Detection

The paper introduces “KoLing”, a bias‑controlled adversarial benchmark of 50.9k Korean instances designed to suppress superficial cues while preserving semantic content, and evaluates both LLM‑generated and LLM‑modified text detection. It also proposes “KoLingProbe”, an interpretable detector grounded in theoretical linguistics that models Korean‑specific properties across multiple linguistic levels. Experiments demonstrate that KoLingProbe outperforms strong baselines on all detection tasks and remains robust to adversarial attacks, with analyses showing that integrating complementary linguistic features across multiple levels improves both performance and interpretability.

By SungHo Kim, SangKeun Lee
Hugging Face Trending Papers
4d ago

Specificity Bias and the Wikipedia Paradox: An xAI Analysis of AI Text Detectors under Domain Shift

The paper examines how fine‑tuned Transformer detectors (DistilBERT, RoBERTa) perform on an out‑of‑distribution dataset of hyper‑local news generated by LLMs. Using SHAP‑based explainable AI, it identifies three phenomena: a reliance on a “Call‑to‑Action” syndrome, a “Specificity Bias” where hallucinated named entities evade detection, and a “Wikipedia Paradox” where formal human text is misclassified as AI. These results highlight systematic shortcomings of purely statistical detection methods in structured human domains.

arXiv Computation and Language
4d ago

TRIDENT and SHAKTI: A Framework and Benchmark for Hindi AI-Text Detection

The paper introduces SHAKTI, a large-scale Hindi benchmark for detecting AI-generated text, covering multiple genres, LLM families, prompting strategies, and adversarial transformations. It evaluates the benchmark with human annotation and examines linguistic, grammatical, and sentiment differences between human and AI texts. Building on these insights, the authors propose TRIDENT, a round‑trip translation‑based detection framework with three variants that outperforms existing baselines and demonstrates robustness across seven languages and adversarial settings.

By Shifali Agrahari, Sujit Kumar, Sanasam Ranbir Singh