← Back to all news
arXiv AI By Rohan Alexander

Iterated creation, grading, and revision of data science projects by language models

Read the original on arXiv AI →

The study evaluates how agentic AI models can autonomously complete end‑to‑end data science projects and improve through iterative feedback. A writer model generates a full project—from data acquisition to analysis and paper writing—while a grader model evaluates it; this cycle repeats for up to three rounds. Across 49 writer‑grader pairs from five developers, initial submissions improved from an average of 59 % to 76 % after feedback, with variations depending on the specific model roles.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 19

EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement?

EvalResearchBench (ERB) is a benchmark that tests whether AI agents can autonomously design evaluation tasks for recursive self‑improvement. In ERB, a researcher agent selects or synthesizes tasks, builds graders, and refines them through pilot tests before producing a final evaluator that runs within fixed time and API budgets. Experiments with nine researchers and thirteen candidate models show that the best automated evaluators achieve about 75% concordance with target benchmarks, below the 91% ceiling set by target disagreements, and that human‑designed task samples still outperform many automated approaches.

By Yaolun Zhang, Tianyi Xu, Yujie Zhao, Jishen Zhao, Qingyun Wu, Huazheng Wang
agentsbenchmarks
More like this →
arXiv AI
Aug 19

Grading Needs a Rubric, Not Intelligence

Small language models can grade open‑ended exam answers as reliably as much larger models when they use an explicit rubric. In experiments with six cost‑efficient model configurations, the rubric decouples grading from judge intelligence, with answer identity explaining 95.6% of score variance and judge identity only 0.2%. Removing rubric criteria or the official answer collapses reliability and inflates scores, showing the rubric’s essential role.

By Jhen-Ke Lin
llms
More like this →