Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

22,530 stories · RSS feed

arXiv AI
Jun 9

NutriMLLM: Multimodal Large Language Models for Dietary Micronutrient Analysis

arXiv:2606. 08948v1 Announce Type: cross Abstract: Comprehensive estimation of dietary micronutrients from food images could improve clinical nutrition care, but training such models requires large multimodal datasets linking diverse foods to complete nutrient profiles.

By Runze Yan, Minxiao Wang, Jiaying Lu, Darren Liu, Xiao Hu, Hanqi Luo
arXiv AI
Jun 9

From USD Scenes to Knowledge Graphs: Zero-Shot Ontology Grounding with LLMs

arXiv:2606. 09134v1 Announce Type: cross Abstract: Constructing knowledge graphs from 3D simulation scenes is essential for robot task reasoning, but the key bottleneck, grounding scene objects to formal ontology classes, still relies on manually curated dictionaries that are brittle and do not generalize across assets.

By Jiangtao Shuai, Zongxiong Chen, Manfred Hauswirth, Sonja Schimmler
arXiv Machine Learning
Jun 9

Generalization in Nonlinear Least Squares via Learned Feature Geometry

arXiv:2606. 08799v1 Announce Type: cross Abstract: We study the generalization of ridge-regularized nonlinear least-squares models via on-average algorithmic stability, deriving error bounds for local minimizers in terms of a data-dependent effective dimension that reflects the geometry of the gradient model at the trained parameters, through the empirical Jacobian Gram matrix and a residual--curvature term.

By Ayub Kharel, Ilja Kuzborski, Patrick Rebeschini, Yasin Abbasi-Yadkori
arXiv AI
Jun 9

A Dataset for Dynamic Human Preferences for Vision Language Models

arXiv:2606. 07653v1 Announce Type: cross Abstract: Given the increased adoption of Vision Language Models (VLMs) in human-interactive settings, it is important that we evaluate how well these models can adapt to real-time preferences for different users.

By Hannah Gao (Massachusetts Institute of Technology), Dylan Hadfield-Menell (Massachusetts Institute of Technology), Rachel Ma (Massachusetts Institute of Technology)
arXiv AI
Jun 9

From Statute to Control Flow: Span-Grounded Deontic Trees for Defeasible Scope Parsing

arXiv:2606. 08932v1 Announce Type: cross Abstract: Rule-following agents tasked with executing policies and regulations often fail via Silent Scope Omission (SSO): a model applies a general rule but silently drops nested exceptions or counter-exceptions, producing outputs that appear compliant yet break on important edge cases.

By Jian Chen, Siyuan Li, Chucheng Wan, Zixuan Yuan
arXiv AI
Jun 9

HARBOR: A Harness Framework for Agentic Robot Reinforcement Learning

arXiv:2606. 08610v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become a powerful paradigm for robot learning, particularly in sim-to-real settings, but its broader adoption remains limited by the engineering pipeline surrounding the algorithms.

By Zechu Li, Yufeng Jin, Xiaoyang Liu, Puze Liu, Vignesh Prasad, Carlo D'Eramo, Georgia Chalvatzaki