arXiv Computation and Language

Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision

The paper introduces Mr Dre, an evaluation suite that tests Deep Research Agents (DRAs) on multi-turn report revision, a process more reflective of human research practices. It defines a unified long‑form report protocol covering comprehensiveness, factuality, and presentation, and includes a human‑verified feedback simulation pipeline. Analysis of five DRAs shows that while agents can address most user feedback, they regress on 16‑28% of previously covered content and citation quality, and over successive turns they disrupt unrelated content and fail to preserve earlier edits, with prompt engineering and dedicated subagents offering limited improvement.

arXiv AI
Sep 1

Redesigning and Auditing Deep Research Writing for Faithful Reports

The paper introduces CLAIMPROBE, a claim-level audit tool that breaks down deep-research reports into individual claims and evaluates them for hallucination, misattribution, citation hygiene, and necessary-fact recall against retrieved evidence. Using CLAIMPROBE, the authors show that even high-scoring deep-research pipelines can omit key evidence and misattribute claims. They also propose CLAIMWRITER, a hierarchical claim-based writer that extracts source facts, maps them to an outline, and drafts sections from a source-linked claim representation, which reduces hallucination by 2.6 to 4.5 times and improves necessary-fact recall by 1.2 to 1.7 times while preserving overall report quality and enabling efficient localized revisions.

By Hiroaki Hayashi, Pranav Narayanan Venkit, Prafulla Kumar Choubey, Chien-Sheng Wu
arXiv AI
Jul 8

Prompt-to-Paper: Agentic AI System for Bioinformatics

arXiv:2607. 05456v1 Announce Type: new Abstract: While recent advances in large language models have enabled end-to-end automated manuscript generation, existing systems suffer from three critical deficiencies: (i) generated claims are not deterministically grounded in verifiable literature, (ii) experimental results are frequently fabricated rather than executed, and (iii) there exists no standardized, multi-dimensional framework to assess whether AI-generated manuscripts meet the quality and rigor required for real-world publication.

By Ramsha Kamran, Maheera Amjad, Zartasha Mustansar, Arsalan Shaukat, Salma Sherbaz, Muhammad U. S. Khan
arXiv AI
Sep 30

LongCat-DeepResearch Technical Report

LongCat-DeepResearch is a deep research system that merges an enhanced LongCat model with a multi‑agent workflow to produce comprehensive, evidence‑grounded reports. The workflow separates global planning from detailed investigation, using planning agents to create a ResearchSpec and research agents to draft sections in parallel, followed by targeted local revisions guided by global review. The system achieves strong benchmark scores, including 55.25 on DeepResearchBench and 79.83 on ResearchRubrics, and shows benefits from combining planning perspectives and additional editing for readability.

By Meituan LongCat Team, He Zhu, Yue Xu, Wanli Wu, Haolin Ren, Yuxin Bian, Jiarui Zhao, Rongzhi Zhang, Quanchi Weng, Jinghao Cui, Yu Fan, Yuhan Liu, Yunhu Ye, Jiyuan Ren, Fengcheng Yuan, Zhao Yang, Jiacheng Zhang, Yuchuan Dai, Ruixuan Xiao, Haozhe Sun, Xiangyuan Liu, Cheng Sun, Yao Du, Yiming Hao, Hongbo Guo, Shuo He, Lei Wang, Xunliang Cai, Yan Chen, Fan Yang, Lingchuan Liu
arXiv AI
Jun 4

Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation

arXiv:2605. 29861v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthesizes scattered evidence into long-form reports.

By Chenghao Zhang, Guanting Dong, Yufan Liu, Tong Zhao, Xiaoxi Li, Zhicheng Dou
arXiv AI
Jul 1

FARS: A Fully Automated Research System Deployed at Scale

arXiv:2606. 31651v1 Announce Type: new Abstract: Recent automated research systems show that language-model agents can generate hypotheses, run experiments, and write complete manuscripts, but most evidence still comes from selected examples, human-framed topics, or a few pre-defined research tasks.

By Qiong Tang, Xiangkun Hu, Xiangyang Liu, Yiran Chen, Yunfan Shao
arXiv Machine Learning
Aug 24

Metag: A dataset to build agentic meta-reviewing capabilities

arXiv:2608.20488v1 Announce Type: new Abstract: AI tools increasingly support tasks across the scientific research cycle, from experiment design and manuscript preparation to peer review. At the same...

By Anirudh Sundar, Min Chen, Divya Tadimeti, Gemma Zhang, Alice Li, Nigel Boachie Kumankumah, Pavan Uttej Ravva, Sadid Hasan, Somya Chatterjee, Pruthvi Prakash Navada, Xiao Wang, Yue Kang, Sulaiman Vesal, Larry Heck
arXiv Computation and Language
Oct 8

Goldsmith: Gold-Loss-Guided Definition Optimization with an Agentic Annotation Harness

Goldsmith is an agentic pipeline that transforms a small set of expert-annotated calibration examples into a reusable structured annotation definition treated as a trainable textual object. Candidate definitions are evaluated on the same gold examples using an executable structured loss, and a large language model editor revises high-loss failures only when loss decreases. The approach outperforms direct rewriting, OPRO, APE, and PromptBreeder in prompt-optimization tests and enhances downstream annotation across typed span, pair-level relation, and fixed-trigger event-argument tasks when combined with retrieval, score-based routing, and human review.

By Yihan Li, Hanyi Zhang, Xiaoxi Jiang, Man Guo