arXiv AI

SCOPE: Source Credibility and Outlet Profiling Engine

SCOPE is a multi‑agent system that profiles news media outlets for bias, factuality, and credibility by coordinating agents for article acquisition, content analysis, evidence search, scoring, and report generation. It follows an MBFC‑style methodology and provides inspectable intermediate evidence and outputs via a structured interface. On a benchmark of 200 outlets, SCOPE outperforms LLM‑only, article‑only, search‑only, and media‑background‑check baselines, achieving the lowest bias and factuality MAE, perfect failed‑fact‑check detection, and the lowest factual error rate.

arXiv Computation and Language
Sep 18

Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories

The paper introduces Data Journalist Agent (Data2Story), a multi‑agent framework that orchestrates specialized roles into a single virtual newsroom to produce evidence‑grounded, multimodal news stories. It ensures every claim is traceable to data, code, or external references via an Inspector, and generates interactive visualizations such as maps and audio to match reader interests. Evaluations on 18 articles show competitive performance in angle coverage, rubric scores, and verifiability, while human writers still lead in editorial angle and creative design.

By Kevin Qinghong Lin, Batu EI, Yuhong Shi, Pan Lu, Juil Sock, Djordje Padejski, Philip Torr, James Zou
arXiv Computation and Language
Sep 28

Epstein Files Engine: Agentic Search for Investigative Journalism

The Epstein Files Engine is an AI agent developed by the New York Times to help journalists investigate a massive mixed‑media collection released by the U.S. Department of Justice on January 30, 2026, which contains about three million pages of PDFs related to Jeffrey Epstein. The Engine translates reporter questions into Google BigQuery SQL queries across three corpora—Epstein‑related releases, the Times’s archive, and external Epstein‑related news headlines—using an LLM to plan queries and return citation‑rich answers that reporters can verify. Over 100 journalists used the Engine, contributing to at least 20 published stories, and the system includes a Diff method for text‑and‑visual duplicate matching to surface genuinely new information.

By Duy K. Nguyen, Teresa Mondr\'ia Terol, Dylan Freedman, Zach Seward
arXiv AI
Oct 27

Same Story, Different Lens: A Multi-Agent System for Cross-Outlet News Framing Analysis

Same Story, Different Lens is an experimental prototype that compares how 17 news outlets cover the same events by collecting world and politics coverage, grouping reports about identical incidents, and generating structured analyses of sentiment, political lean, framing, loaded language, omissions, and estimated proportions of factual claims versus opinion. The system employs three model roles—article analysis, source support assessment, and cross‑outlet briefing generation—to produce JSONL archives of its findings. Evaluation includes comparing outlet‑level lean estimates with predefined reference ratings and summarizing the Critic’s grounding scores, which indicate the model’s assessment of support within the supplied text.

By Affan Yasin, Yueqi Lyu, Rubia Fatima, Weinchen Zhong, Keyi Li
arXiv AI
Oct 28

Beyond the Evidence Coherence Trap: Agent-Driven Evidence Selection for Temporally Disordered Fake News Detection

The paper introduces ChronoFake, a benchmark of 30,706 news articles that isolates temporally disordered fake news, where factual events are rearranged into false chronologies. It identifies the "evidence coherence trap," where verification relies on locally coherent but temporally mismatched evidence. To counter this, the authors propose DelSA, a document-selection agent that iteratively refines evidence during training and uses a frozen rulebook for inference, leveraging an LLM for evidence characterization and a verifier-trained reranker for timeline consistency, achieving superior performance on ChronoFake and competitive results on other datasets.

By Chi Wang, Min Gao, Yifei Zhang, Bincheng Gu, Zongwei Wang, Wentao Li
arXiv Computation and Language
Oct 23

Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports

The paper introduces Video‑Centered Deep Research Report Generation, a new benchmark that evaluates agents on their ability to generate long, interactive research reports from a source video and user request. It presents the OmniDR‑Bench dataset with 50 tasks in Chinese and English, 19 subcategories, and 6,414 criteria, and proposes Omni Atomic Rubric Trees (Omni‑ARTs) for evaluation across audiovisual grounding, multimodal evidence use, and interactive presentation. The authors also detail a skill‑based workflow, an automated rubric construction process, and report that current agents achieve a best score of 63.3/100, highlighting gaps in content coverage, causal analysis, and citation accuracy.

By Jiahao Meng, Qize Yang, Yuxuan Wang, Yunfei Chu, Wensi Dai, Jin Xu, Yunhai Tong
arXiv AI
Jun 19

Benchmarking Agentic Review Systems

arXiv:2606. 19749v1 Announce Type: new Abstract: A new class of agentic review systems are emerging as a remedy to the pressure placed on peer review systems by AI-assisted research, but it is unclear how they should be evaluated.

By Dang Nguyen, Wanqing Hao, Yanai Elazar, Chenhao Tan