arXiv Computation and Language

Generating Edit-Inducing Questions for AI Research Manuscripts

arXiv Machine Learning
6d ago

Edit Discovery in Human-AI Interactions

arXiv:2610.22661v1 Announce Type: new Abstract: Human-AI interactions are becoming a rich source of data for improving AI systems: while many model responses are already satisfactory, others expose f...

By Alireza Mirrokni, Shayan Kiyani, George Pappas, Hamed Hassani
arXiv AI
Jul 1

Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA

arXiv:2606. 32002v1 Announce Type: new Abstract: Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the resulting pairs are used to fine-tune, distill, or compress knowledge into another model.

By Ekaterina Alimaskina, Denis Shveykin, Gleb Molodtsov, Igor Shalygin, Alexey Kadeishvili, Aleksandr Beznosikov
arXiv AI
Sep 24

Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation

The paper proposes using large language models (LLMs) to identify disagreements among models as a way to focus expert effort on revising codebooks for large‑scale text annotation. Three expert feedback methods are evaluated: editing LLM‑generated revisions (Codebook Verifying), answering questions about disagreements (Question Answering), and labeling disagreement cases with rationales (Rationale Labeling). Experiments on tutoring‑session transcripts show that Rationale Labeling achieves the highest LLM‑labeling accuracy (64.9%) compared to the expert‑revised codebook (57.8%), with Question Answering also outperforming the baseline (60.5%).

By Zeyu He, Zhuqian Zhou, Kirk Vanacore, Rene F. Kizilcec, Ting-Hao 'Kenneth' Huang
arXiv Computation and Language
Sep 17

Reading Between the Lines: Can LLMs Discover the Question Behind the Text?

The paper introduces "question archaeology," an evaluation task that asks models to infer the single, authentic question that motivated a text. It presents a new dataset of commissioned texts paired with their original research questions and distractors, and evaluates both proprietary and open‑source LLMs. Results show newer models outperform older ones, with BERT-based models lagging, and current LLMs even surpassing human performance on this task.

By Claudiu Creanga, Liviu P. Dinu
arXiv AI
Aug 28

How LLMs Distort Our Written Language

Large language models (LLMs) are widely used to assist writing, but this study shows they alter both tone and meaning of human text. A user study found that heavy LLM use increased neutral essays by nearly 70% and made writers feel less creative and less in their voice. Even when prompted to make only grammar edits, LLMs changed the semantic content of essays and produced AI-generated scientific reviews that were less focused on clarity and significance and scored higher on average.

By Marwa Abdulhai, Isadora White, Yanming Wan, Ibrahim Qureshi, Joel Z. Leibo, Max Kleiman-Weiner, Natasha Jaques