Hugging Face Blog
What We Learned by Reproducing 2,200 papers from ICML
Read the original on Hugging Face Blog →The Flow has not summarised this story yet — read it at Hugging Face Blog.
Related stories
Sebastian Raschka
Jul 1, 2025
LLM Research Papers: The 2025 List (January to June)
A topic-organized collection of 200+ LLM research papers from 2025
Sebastian Raschka
Jun 6
LLM Research Papers: The 2026 List (January to May)
A curated roundup of notable LLM research papers that came out this year
arXiv Machine Learning
Sep 2
Modelpedia: A Catalog of Model Findings for the Meta-Science of AI
Modelpedia is an automated, LLM-assisted framework that extracts and organizes findings about AI models from published papers into a searchable public catalog. It links each finding to the relevant model, dataset, method, and concept, and has already extracted over a thousand findings from ICLR 2024 and 2025 papers. The authors invite the community to explore, contribute to, and build on this open catalog, positioning model findings as a shared foundation for the meta‑science of AI.
arXiv AI
6d ago
Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results
arXiv:2604.21965v2 Announce Type: replace Abstract: Recent work has used LLM agents to reproduce empirical social science results with access to both the data and code. We broaden this scope by askin...
arXiv Machine Learning
Jul 31
AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis
arXiv:2607. 28618v1 Announce Type: cross Abstract: Chemistry literature synthesis often requires assembling specific findings scattered across many publications, yet existing literature-search systems primarily return ranked document lists.
arXiv Computation and Language
Sep 11
Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers
The article introduces AgentActionBench, a benchmark designed to evaluate agent-based experiment reproduction across machine learning and AI4Science papers. It employs an MCP-based Action Recorder to capture agents’ behavior during reproduction and assesses the resulting traces against paper-specific rubrics. The benchmark includes 150 papers, with a human-annotated subset and model-assisted augmentation expanding it to over 10,000 rubric items, revealing that current systems face execution bottlenecks but that model-generated rubrics correlate strongly with human judgments.