arXiv AI

Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens

arXiv AI
3d ago

Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems

The paper "Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems" updates the 2024 survey on generative AI and misinformation, shifting from a literature review to practical countermeasures. It documents changes in the threat landscape, noting that Large Language Models and multimodal systems have improved AI-generated content, thereby worsening information quality. The authors introduce JudgeGPT, a platform for assessing human perception of AI-generated news, and RogueGPT, a controlled stimulus generation engine, forming an experimental pipeline to study detection and mitigation strategies, including LLM-based detection and inoculation approaches.

By Alexander Loth, Martin Kappes, Marc-Oliver Pahl
arXiv Computation and Language
1d ago

Eroding the Truth-Default: A Causal Analysis of Human Susceptibility to Foundation Model Hallucinations and Disinformation in the Wild

The paper introduces JudgeGPT and RogueGPT, a dual‑axis framework that separates authenticity from attribution to study how people detect hallucinations and disinformation from foundation models. Using 918 evaluations on five models, including GPT‑4 and Llama‑2, the authors apply Structural Causal Models to test causal hypotheses about detection accuracy. Results show political orientation has little effect, while familiarity with fake news mediates detection, and a "fluency trap" in GPT‑4 outputs makes them hard to distinguish from human text, implying pre‑bunking should focus on cognitive source monitoring.

By Alexander Loth, Martin Kappes, Marc-Oliver Pahl
arXiv AI
Sep 10

CogniDir: Combating Cognitive Malicious Comments via Adaptive Distributional Learning for Robust Fake News Detection

CogniDir is an adaptive distributional learning framework designed to improve fake news detection against new psychologically grounded malicious comments generated by Large Language Models. It reframes robust detection as a dynamic data mixture optimization problem, using cognitive psychology to formalize adversarial paradigms and an information‑theoretic score to guide adaptive sampling of training data. Experiments on three benchmarks show that CogniDir achieves state‑of‑the‑art robustness, boosting F1 scores by up to 17.9% over existing baselines under heterogeneous AI‑generated attacks.

By Zhao Tong, Chunlin Gong, Yimeng Gu, Haichao Shi, Qiang Liu, Shu Wu, Xingcheng Xu, Xiao-Yu Zhang
arXiv AI
Aug 25

AI Watchdog: Agent Interfaces for Detecting and Defending Against Manipulative Dark Patterns in AI Conversations

arXiv:2608.21841v1 Announce Type: new Abstract: Conversational AI increasingly shapes consequential decisions, yet users have limited support for recognizing and resisting manipulation. We present AI...

By Rachel Poonsiriwong (Pub), Chayapatr (Pub), Archiwaranguprok, Constanze Albrecht, Monchai Lertsutthiwong, Pattie Maes, Pat Pataranutaporn
arXiv AI
3d ago

The Verification Crisis: Expert Perceptions of GenAI Disinformation and the Case for Reproducible Provenance

The article reports on a survey of 21 experts—including AI researchers, policymakers, and disinformation specialists—about the risks posed by Generative Artificial Intelligence (GenAI) to information integrity. It finds that while deepfake videos are immediately shocking, large‑scale text generation threatens to create systemic epistemic fragmentation and synthetic consensus, especially in politics. Experts express doubt about current detection tools and instead advocate for provenance standards and regulatory frameworks, emphasizing the need for reproducible research methods and standardized benchmarks to effectively counter synthetic media.

By Alexander Loth, Martin Kappes, Marc-Oliver Pahl
arXiv AI
Oct 15

Hidden in Plain Sight: Advancing Harmful Content Understanding via Perception-Aware Adversarial Self-Play

arXiv:2610.15449v2 Announce Type: new Abstract: Despite significant advances in language understanding, Large Language Models (LLMs) can still miss meaning hidden in plain sight, failing to identify...

By Yisong Xiao, Benlei Cui, Haiwen Hong, Shiji Zhao, Zonghao Ying, Yaoyuan Zhang, Longtao Huang, Aishan Liu, Hui Xue, Ting Ma, Xianglong Liu