Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

12,573 stories · RSS feed

arXiv Machine Learning
4d ago

Advancing Open and Reproducible Relational Learning: RelArena-$\alpha$, TabPFN-Rel and RPI

arXiv:2608. 16319v1 Announce Type: new Abstract: This first release of Prior Labs in relational learning shows our continued commitment to open science.

By Adrian Hayler, Klemens Fl\"oge, Alan Arazi, Rishabh Ranjan, Jure Leskovec, Felix Birkel, Brendan Roof, Anurag Garg, Kristina Collins, Lydia Sidhoum, Jonas K\"ubler, Siyuan Guo, Oscar Key, Jan Hendrik Metzen, Rylee Grace, David Salinas, Arthur Cahu, Simon Bing, Benjamin J\"ager, Tuana \c{C}elik, Mihir Manium, Vitor Monteiro, Jake Robertson, Jerry Chen, Eliott Kalfon, Tom\'as Pereda, Lilly Wehrhahn, Dominik Safaric, Tobias Schroeder, Georg Grab, Diana Kriuchkova, Clara Cornu, Philipp Singer, Nick Erickson, Vahid Balazadeh, Marie Salmon, Simone Alessi, K\"ur\c{s}at Kaya, Philipp Jund, L\'eo Grinsztajn, Yann LeCun, Bernhard Sch\"olkopf, Madelon Hulsebos, Lennart Purucker, Sauraj Gambhir, Frank Hutter, Noah Hollmann
arXiv Machine Learning
4d ago

Mint-Agent: Introducing Finance-Native Agentic Foundation Models

arXiv:2608. 16386v1 Announce Type: cross Abstract: Financial agents must do more than recall domain knowledge: they must be both reliable, executing precise operations over grounded evidence, and executive, sustaining long-horizon research whose conclusions remain auditable.

By Agent Team, B. Zhang, Yaze Geng, Lei Tang, Yaoyang Yi, Zonghan Wu, Yifan Hu, Kun Wang, Qingsong Wen, Yilei Shao
arXiv Machine Learning
4d ago

Convex Optimization with Nested Evolving Feasible Sets

arXiv:2605. 07386v2 Announce Type: replace Abstract: \emph{Convex Optimization with Nested Evolving Feasible Sets (CONES)} is considered where the objective function \(f\) remains fixed but the feasible region evolves over time as a nested sequence \(S_1 \supseteq S_2 \supseteq \cdots \supseteq S_T\).

By Karthick Krishna M., Haricharan Balasundaram, Rahul Vaze
arXiv AI
4d ago

Personalized Auto-Research: Towards a True AI Co-Scientist

arXiv:2608. 14881v1 Announce Type: new Abstract: AI co-scientists that generate hypotheses, retrieve related work, design experiments, execute code, and draft full papers are beginning to change how research is carried out.

By Bo Ni, Franck Dernoncourt, Hongjie Chen, Yu Wang, Nesreen K. Ahmed, Zhengzhong Tu, Tyler Derr, Ryan A. Rossi