The paper investigates how large language models (LLMs) can perform multi-coder qualitative coding by independently coding, debating, and reconciling disagreements. It quantifies the effectiveness of this approach across diverse datasets, identifying key factors—such as codebook length, data similarity, and agent disagreement—that influence coding accuracy. The study finds that intense, unresolved debates improve accuracy but that LLMs still lack adaptive responsiveness to context, leading to design recommendations for automated coding systems.
By Jeongyeon Kim, John Mitchell
QuAD is a framework that quantitatively assesses the quality of qualitative response datasets before analysis. It compares an intrinsic cluster distribution derived from the responses with an extrinsic, researcher-defined analytical lens based on metadata, using a four‑quadrant assessment that considers lens‑level relations and within‑cluster variation. The method employs a content‑aware multi‑agent system with two‑stage instruction generation and adaptive anchors to improve scoring consistency, and is validated through synthetic experiments and a real‑world social survey case study.
By Wenshuo Zhang, Leixian Shen, Qiqi Zhou, Shuchang Xu, Feiyang Hao, Huamin Qu, Haobo Li, Zhao Chen, Lin-Ping Yuan
arXiv:2601. 14171v2 Announce Type: replace Abstract: Writing effective rebuttals is a high-stakes task that demands more than linguistic fluency, as it requires precise alignment between reviewer intent and manuscript details.
By Qianli Ma, Chang Guo, Zhiheng Tian, Siyu Wang, Jipeng Xiao, Yuanhao Yue, Zhipeng Zhang
arXiv:2608.30543v1 Announce Type: new
Abstract: Large Language Models (LLMs) offer new possibilities for scaling qualitative analysis, but existing applications often provide limited methodological t...
By Nadia Jul Jeldtoft, Tariq Yousef
DataSTORM is an LLM‑based agentic system designed to conduct deep research over large‑scale structured databases and internet sources. It applies principles of Exploratory Data Analysis and Data Storytelling to frame research as a thesis‑driven analytical process, iteratively generating hypotheses, performing quantitative reasoning, and crafting coherent narratives. Evaluations on InsightBench and a new ACLED‑based dataset show that DataSTORM surpasses existing systems, achieving significant improvements in insight‑level recall and summary‑level scores.
By Shicheng Liu, Yucheng Jiang, Sajid Farook, Camila Nicollier Sanchez, David Fernando Castro Pena, Monica S. Lam
arXiv:2607. 28889v1 Announce Type: cross Abstract: Qualitative researchers increasingly encounter interaction corpora whose scale exceeds what manual coding alone can address, and large language models (LLMs) are frequently proposed as analytic assistants.
By Alex Liu, Min Sun, Lief Esbenshade, Michael Xiao, Victor Tian, Zachary Zhang, Kevin He
The paper introduces Data Journalist Agent (Data2Story), a multi‑agent framework that orchestrates specialized roles into a single virtual newsroom to produce evidence‑grounded, multimodal news stories. It ensures every claim is traceable to data, code, or external references via an Inspector, and generates interactive visualizations such as maps and audio to match reader interests. Evaluations on 18 articles show competitive performance in angle coverage, rubric scores, and verifiability, while human writers still lead in editorial angle and creative design.
By Kevin Qinghong Lin, Batu EI, Yuhong Shi, Pan Lu, Juil Sock, Djordje Padejski, Philip Torr, James Zou
arXiv:2610.08364v1 Announce Type: new
Abstract: Frontier AI evaluations increasingly use open-ended, agentic, long-horizon tasks whose transcripts can span hundreds of pages of outputs and actions fr...
By Toby D. Pilditch, Konstantinos Voudouris, Alexandra Abbas, Cozmin Ududec
arXiv:2610.02679v1 Announce Type: new
Abstract: Data journalism, the practice of using data analysis to surface newsworthy stories, depends increasingly on the ability of reporters and investigative...
By Raquib Bin Yousuf, Harith Laxman, Vitaliy Shkremetko, Eunice Son, Shambhavi Verma, Brian O'Leary, Venketesh Subramony, Sylvain Nazef, Jacquelyn Elias, Ron Coddington, Chris Contakes, Michael Riley, Naren Ramakrishnan
arXiv:2606. 06025v1 Announce Type: cross Abstract: Scientific peer review generation has attracted increasing attention for reducing reviewing burdens and providing timely feedback.
By Xinpeng Qiu, Wang Yihu, Zhifeng Liu, Xiaochen Wang, Jimin Wang
arXiv:2607. 18816v1 Announce Type: cross Abstract: LLM-powered agents increasingly tackle complex tasks by invoking tools, querying databases, executing code, and manipulating intermediate artifacts.
By Eden Wu, Sonia Castelo, Yurong Liu, Cl\'audio T. Silva, Juliana Freire
The paper introduces an author‑in‑the‑loop collaborative audit system for reproducibility checks during peer review, where authors explicitly declare reproduction instructions via a structured checklist. An AI agent uses these declarations to investigate and assess reproducibility, while human reviewers approve requests and decide on issue resolution. The system was deployed on 36 conference papers, identifying 20 issues that were subsequently resolved, and the study examined how different checklist conditions affected the agent’s assessment.
By Yukun Dong, Linjun Zhang, Tian Zheng