The paper proposes a framework that adapts systematic review methods—planning, conducting, and documenting—to the retrieval of machine learning assets in AI registries. By treating models, datasets, and benchmarks as first‑class units of analysis, the framework incorporates registry‑aware search, cross‑registry schema alignment, and dependency‑driven exploration to make asset selection transparent, reproducible, and evidence‑based. The authors argue that this approach transforms ML asset discovery from an informal, ad‑hoc process into a structured, systematic practice.
arXiv:2607. 17242v1 Announce Type: cross Abstract: Pretrained machine learning (ML) models help developers build ML-intensive software systems without training models from scratch.
Modelpedia is an automated, LLM-assisted framework that extracts and organizes findings about AI models from published papers into a searchable public catalog. It links each finding to the relevant model, dataset, method, and concept, and has already extracted over a thousand findings from ICLR 2024 and 2025 papers. The authors invite the community to explore, contribute to, and build on this open catalog, positioning model findings as a shared foundation for the meta‑science of AI.
Benchmark Radar is a living database and search engine that aggregates AI benchmark papers, datasets, code, and score histories. It automatically discovers new benchmark resources from 37 sources, maintains a catalog of 1,283 records with 12,916 numeric observations, and provides tools such as a web dashboard, CLI, and downloadable evidence for researchers. The system also offers visualizations like a Pareto frontier and trend views to help users assess benchmark saturation and adoption.
arXiv:2607. 16845v1 Announce Type: new Abstract: Scientists at European XFEL conduct experiments that generate very large and complex datasets.
arXiv:2608. 19625v1 Announce Type: new Abstract: Scientific data are increasingly used by AI agents, yet existing dataset representations provide limited support for autonomous discovery, interpretation, and invocation.
arXiv:2610.07132v1 Announce Type: cross Abstract: Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful rea...
arXiv:2606. 00644v1 Announce Type: new Abstract: AI research often requires decisions before future evidence exists: which bottleneck to attack, which direction to pursue, or where a project should be positioned.
arXiv:2606. 09809v1 Announce Type: new Abstract: AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs.
The paper introduces the Scientific Data Skill (SciDSK), an agent‑ready representation that packages dataset‑specific knowledge and operational guidance as a reusable skill. SciDSK integrates dataset descriptions, scientific context, file organization, usage procedures, quality checks, and provenance information while keeping the data in its original repository. The authors define a structured specification, build a construction pipeline, and launch the Scientific Data Skill Bank to publish SciDSK resources across six scientific disciplines, demonstrating improved agent‑driven dataset discovery and interpretation through evaluation benchmarks.
arXiv:2604. 10311v2 Announce Type: replace Abstract: Artificial Intelligence (AI) models, encompassing both traditional machine learning (ML) and more advanced approaches such as deep learning and large language models (LLMs), play a central role in modern applications.
arXiv:2606. 31366v1 Announce Type: cross Abstract: Driven by high-throughput experimentation, computational modeling, and artificial intelligence (AI), materials data has expanded at an unprecedented rate.
The article proposes a framework called quantitative evidence mining to transform biomedical findings into structured, context-rich evidence units. It outlines core elements such as claim, measured entity, value, comparator, population, conditions, temporal context, uncertainty, provenance, validation, and expert review. The authors present an eight-stage reference architecture and emphasize that plausibility should remain multidimensional rather than collapsed into a single truth label, linking extraction to evidence synthesis for applications like clinical trials, biomarker research, and knowledge-graph construction.