arXiv Computation and Language

Eroding the Truth-Default: A Causal Analysis of Human Susceptibility to Foundation Model Hallucinations and Disinformation in the Wild

The paper introduces JudgeGPT and RogueGPT, a dual‑axis framework that separates authenticity from attribution to study how people detect hallucinations and disinformation from foundation models. Using 918 evaluations on five models, including GPT‑4 and Llama‑2, the authors apply Structural Causal Models to test causal hypotheses about detection accuracy. Results show political orientation has little effect, while familiarity with fake news mediates detection, and a "fluency trap" in GPT‑4 outputs makes them hard to distinguish from human text, implying pre‑bunking should focus on cognitive source monitoring.

arXiv Computation and Language
1d ago

Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News

Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News investigates whether people can distinguish news articles written by humans from those generated by large language models (LLMs). Using JudgeGPT, the study collected 2,318 judgments from 1,054 participants on content from six LLMs, finding that participants could not reliably differentiate machine-generated from human-written text. The research also shows that domain expertise predicts judgment accuracy, distinct response strategies exist, and accuracy declines after about 30 evaluations due to cognitive fatigue.

By Alexander Loth, Martin Kappes, Marc-Oliver Pahl
arXiv AI
Oct 9

Caught in the Act: Probes Effectively Detect Sabotage and Catch Unverbalized Deception

The paper presents a white‑box deception detection system that uses probes trained on the largest deception dataset (FIBS) and a novel architecture aggregating across layers and tokens. The probes achieve 98.8% AUC on SHADE‑Arena, outperforming a text‑monitoring baseline, and reach up to 99.7% AUC when detecting introspective deception that cannot be inferred from context alone. They also effectively detect deception in open‑weight models on politically sensitive topics and when models are pressured about their beliefs.

By Oskar J. Hollinsworth, Alex F. Spies, Tigist Diriba, Adam Gleave, Chris Cundy
arXiv Computation and Language
Sep 18

FakeSpotter: A content and strategy agnostic Viral Misinformation Detection Tool

FakeSpotter is a new tool that estimates the viral misinformation risk of textual content by measuring structural fingerprints of misinformation instead of directly judging truthfulness. It operates across linguistic, narrative, logical, and critical‑thinking dimensions, using repeated large language model assessments and domain‑specific logistic regression classifiers for both short and long texts. In a labeled corpus of 764 texts, FakeSpotter achieved macro F1 scores of 0.788 for short texts and 0.793 for long texts, and its interpretive layer offers explainable outputs such as feature‑based scores, signal agreement, and a caution index for social listening.

By Giovanni Spitale, Federico Germani
arXiv Machine Learning
Sep 29

Fake News Theories: Harnessing Disciplinary Insights for Computational Modeling, Detection, and Explanation

The paper presents a theory-informed computational framework that converts cross-disciplinary theories of fake news into measurable features for automated detection and explanation. By reviewing theories from social sciences, psychology, economics, and more, the authors establish a broad theoretical foundation for computational modeling. Experiments on benchmark datasets demonstrate that theory-derived features are predictive, provide interpretable diagnostic signals, and that multi-feature models generally outperform individual features, though gains are modest.

By Zhaoyang Cao, Miriam Metzger, Reza Zafarani
arXiv AI
Oct 19

Validating LLM-generated political networks with survey-reported partisan exposure

The study evaluates large language models (LLMs) that generate political networks by comparing them to survey-reported partisan exposure from the ANES 2020–2022 Social Media Study. It finds that while network structure and political assortativity vary across model configurations, many models either over‑separate Democratic and Republican egos or underestimate Republican exposure, leading to group‑specific errors in local partisan environments. These discrepancies persist even among networks with similar aggregate properties, indicating that aggregate metrics alone can mask important individual‑level inaccuracies.

By Marco Tonin, Michele Tizzoni, Bruno Lepri, Luca Maria Aiello