arXiv:2607. 21842v1 Announce Type: cross Abstract: Research on political polarization on social media depends on the ability to reliably measure partisanship in user-generated content.
By Fathima Ameen, Christopher G. Healey
The paper presents a red‑teaming framework to evaluate how open‑source large language models (LLMs) can be used for political influence, focusing on models that can be deployed locally without weight modifications. It measures LLM Overton Windows (OWs) to assess expression fidelity of sampled political opinions and examines how natural‑language jailbreaks alter these scores. The study evaluates over 30 LLMs from 10 families across five countries, finding that left‑leaning content is expressed with higher fidelity, larger models tend to have lower mean OW scores, and jailbreak effectiveness varies significantly between model families.
By Daniel C. Ruiz, Anna Serbina, Ashwin Rao, Emilio Ferrara, Luca Luceri
The paper investigates how to detect enthymemes—arguments with unstated premises or conclusions—in controversial political tweets about vaccination and immigration. Despite extensive fine‑tuning, models achieve an F1 score of only about 0.67, prompting the authors to propose new evaluation metrics that compare predictions to the full distribution of annotator votes and a model that directly predicts this distribution. The study also shows that models rely on explicit argumentative structure, often missing implicit links that readers must infer, raising concerns for content moderation systems that depend on majority labels.
By Martial Pastor, Nelleke Oostdijk
The paper investigates why misinformation spreads more quickly on engagement‑based platforms by dissecting the recommendation algorithm of X. It identifies an engagement fungibility mechanism that rewards instant reactions (likes, retweets) over thoughtful engagement (replies, quotes), allowing misinformation—which tends to attract instant reactions—to receive more recommendations. The authors validate this mechanism through a simulation on the USC X 2024 election corpus, showing that adjusting metric weights has little effect, while requiring thoughtful engagement before amplification can significantly reduce the credibility exposure gap without harming mainstream content or engagement.
By Pan Li, Shuang Gao
The study introduces a computational method to distinguish between antagonizing and criticizing rhetoric in political discourse on Twitter/X. Analyzing over 17 million tweets from German trending topics, it finds that roughly one‑third contain antagonizing rhetoric, which is more prevalent in trending discussions than in general Twitter traffic. Right‑leaning users use antagonizing language more than left‑leaning users, and both forms of rhetoric are amplified within political camps, with certain parties—AfD and the Greens—emerging as primary antagonists.
By Armin Pournaki, Eckehard Olbrich
Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News investigates whether people can distinguish news articles written by humans from those generated by large language models (LLMs). Using JudgeGPT, the study collected 2,318 judgments from 1,054 participants on content from six LLMs, finding that participants could not reliably differentiate machine-generated from human-written text. The research also shows that domain expertise predicts judgment accuracy, distinct response strategies exist, and accuracy declines after about 30 evaluations due to cognitive fatigue.
By Alexander Loth, Martin Kappes, Marc-Oliver Pahl