The study shows that when people enjoy an artwork, they are more likely to believe it was created by a human, regardless of whether it was actually AI-generated or not. In a large online Turing Test, 22.1 percentage points more of respondents who favored a picture thought it was human-made, and this pattern held across all images and detection methods. Even those who were skeptical of AI art often preferred AI-generated pieces but still attributed them to humans, indicating a strong link between aesthetic preference and perceived authorship.
By Zihan Chen, Di Zhu, Lei Nico Zheng
The paper examines whether existing automatic methods can reliably assess creativity in text produced by large language models (LLMs). By collecting human ratings on 11 creativity dimensions for both human and AI short stories, the authors compare these judgments with automated metrics and LLM-as-a-Judge evaluations. The results show a significant misalignment: automated metrics and LLM judges favor AI-generated stories and show near-zero correlation with human assessments, revealing fundamental limitations in current computational approaches to evaluating creative text.
By Alessandro Tutone, Giorgio Franceschelli, Mirco Musolesi
The study investigates bias in large language model (LLM) judges by having ten LLMs evaluate narrative constraint selections rather than generated text. Results show that self-preference largely disappears under blind evaluation when quality and evaluator severity are controlled, but self- and other-labels alone shift scores bidirectionally when quality is matched. The authors conclude that authorship attribution drives evaluation bias and that open-ended, ground‑truth‑free tasks can effectively study LLM judge behavior.
By Songeun Chae, Min Kim, Donghoon Jung, Seojin Choi, Seohyon Jung
However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open question. To address this question, we propose SciStyleBench, a unified three-component benchmark for diagnosing and mitigating stylistic bias in LLM-based idea evaluation: (i) First, SciStyleStage, a three-stage evaluation environment that applies controlled stylistic perturbations to fixed scientific content across three settings no context, fixed-domain context, and open-domain retrieval context, covering 600 scientific ideas and 15 style variants, with 9,000 evaluation instances per setting; (ii) Second, SciStyleMetrics, a set of quantitative measures, including Style Bias Index (SBI), Substance Recognition Rate (SRR), and Adversarial Win Rate (AWR), to characterize how stylistic variation affects scoring stability, substance discrimination, and ranking robustness; (iii) Third, SciStyleExtractor, a plug-and-play evaluation module that separates presentation style from scientific content by predicting style type and deviation before style-conditioned evaluation, enabling us to assess whether style awareness reduces stylistic bias.
arXiv:2608. 01666v2 Announce Type: replace-cross Abstract: However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open question.
By Fengxian Ji, Yuke Li, Jingpu Yang, Juanfan Wu, Fan Zhang, Zhexuan Cui, Yu Xie, Min Peng, Qianqian Xie, Xiuying Chen, Zhuohan Xie
arXiv:2609.15511v1 Announce Type: cross
Abstract: This paper investigates the generation and human evaluation of Japanese haiku by contemporary Large Language Models (LLMs), focusing on authorship pe...
By Livia Oddi, Simone Scardapane, Toru Sugimoto, Donatella Genovese
The paper introduces a new authorship attribution model that represents each author with multiple consistent but distinct writing styles, rather than a single style profile. Experiments across four scenarios—audience size, temporal distance, preceding context, and genre norms—show that authors vary significantly in how idiolectic their writing is, and that certain settings yield text that is more informative about authorship. The authors demonstrate that modeling multiple styles per author leads to significant performance improvements over traditional single-style approaches.
By Omkar Yadav, Michael Jiang, Kenan Alkiek, Jian Zhu, David Jurgens
The paper investigates Self‑Generated Text Recognition (SGTR), the ability of large language models (LLMs) to identify their own outputs. By evaluating 13–21 models across 6 experimental designs, it shows that SGTR accuracy varies with evaluation format, conversation structure, and task domain, and that a quality‑heuristic bias dominates results. The study also finds that fine‑tuning for SGTR in one setting can generalize to others and may cause models to prefer their own outputs when judging, highlighting potential safety concerns.
By Jesse St. Amand, Callum Canavan, Sohaib Imran, Joseph Hewson, Aaron Lutz, Shi Feng, Puria Radmard, Lennie Wells
arXiv:2310.00436v2 Announce Type: replace
Abstract: Authorship identification uses patterns in writing to infer who wrote a text, but those patterns also reflect topic, genre, and register. This surv...
By Haining Wang
The paper argues that current AI‑text detection methods, which estimate human contribution by measuring how predictive a user’s prompt is of the final text, can be easily manipulated. By introducing SPECIFICITY PADDING—adding random IDs to the prompt—the authors show that a title‑only request can appear far more human than one containing the article’s key ideas, and that scores can swing dramatically when the evaluator’s view of the prompt is altered. Across a range of large language models, the same documents can be scored as 85 % human or 15 % human depending solely on the prompt used, revealing a fundamental limitation of likelihood‑based metrics for reliably measuring human authorship.
whyItMatters":"The study demonstrates that likelihood‑based scores cannot reliably establish human contribution without additional evidence of what the person added and how the text was produced."
By Hasan Amin, Zhuoran Lu, Ming Yin
arXiv:2605. 29928v2 Announce Type: replace-cross Abstract: As AI-generated and AI-assisted content floods online spaces, source labels attached to such content can distort human reasoning judgments, with downstream consequences for moderation, evaluation, and decision-making.
By Mahjabin Nahar, Nafis Irtiza Tripto, Aiping Xiong, Ting-Hao 'Kenneth' Huang, Dongwon Lee
arXiv:2607. 21498v1 Announce Type: cross Abstract: A rhetorical figure that Cicero and Quintilian catalogued two thousand years ago reappears, systematically, in the text of large language models: epanorthosis, the self-correction of the specimen {\guillemotleft}This is not a course.
By Federico Boggia