arXiv AI
BAIT: Can LLMs Find What Biases a Reader? A Benchmark for Cognitive Bias Triggers in News
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 24
Definitional Sensitivity in Media Bias Detection: A Multi-Definition Dataset and Benchmark
Media bias detection relies on definitions and examples that specify what counts as bias, yet these specifications often vary across datasets or remain implicit, even when given the same name. Such va...
arXiv Computation and Language
Aug 25
Definitional Sensitivity in Media Bias Detection: A Multi-Definition Dataset and Benchmark
arXiv:2608.23095v1 Announce Type: new Abstract: Media bias detection relies on definitions and examples that specify what counts as bias, yet these specifications often vary across datasets or remain...
arXiv Computation and Language
Sep 25
Framing by Wording, Framing by Selection: A Large-Scale Two-Dimensional Audit of French News Headlines, 2022-2025
The study introduces a two‑dimensional framework to audit French news headlines, distinguishing salience framing—captured by four wording devices—from selection framing—captured by outlet‑level story form and high‑charge distributions. Using a 10,000‑headline supervision set annotated by LLMs and human arbitration, the authors classify 902,111 headlines from 25 outlets (2022‑2025) and find that salience and selection diverge yet correlate, that default thresholds inflate salience estimates, and that headlines mentioning Jews, the Far‑right, and Muslims exhibit the highest salience rates. The authors release their dataset, lexicons, and code, claiming it is the largest French headline framing audit to date.
arXiv AI
Aug 10
Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes
arXiv:2608. 07208v1 Announce Type: cross Abstract: Existing measures of how much a text is about a concept read the surface of the text: dictionary word shares, topic proportions, embedding similarities.
arXiv AI
3d ago
Trapped in Your Story: Evaluating Narrative-Induced Sycophancy in Large Language Models
Large language models (LLMs) tend to exhibit sycophancy by aligning with users’ expressed views. This study introduces a benchmark of 5,500 evaluation instances built from 500 debatable questions across ten domains, where the user stance is fixed but the justification is systematically varied across eleven conditions, including neutral, personal experience, causal framing, and worldview-based synthesis. Seven LLMs were evaluated, revealing that narrative structure can significantly influence agreement, with causal framing and bullet presentation boosting sycophancy while personal experience can reduce it.
arXiv AI
Sep 7
MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate
MABPD (Multi‑Agent Bias Probing & Detection) is a training‑free pipeline that uses three specialized large language model agents to analyze news articles from complementary perspectives and resolve disagreements via a Structured Argument Debate (SAD) protocol. SAD imposes an asymmetric burden of proof—biased claims lacking grounded textual evidence receive zero weight—along with role‑weighted voting and post‑consensus verification, replacing task‑specific supervised decision boundaries. Ablation studies show that the debate module alone accounts for up to a 10.6‑point F1 gain, and on the BABE benchmark MABPD attains 83.4% macro F1, within 0.7 percentage points of the supervised state‑of‑the‑art, while achieving 75.0% zero‑shot accuracy on the SemEval 2019 HyperPartisan corpus.