Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,477 stories · RSS feed

arXiv AI

Jun 30

FlatLands: Generative Floormap Completion From a Single Egocentric View

arXiv:2603. 16016v2 Announce Type: replace-cross Abstract: A single egocentric image typically captures only a small portion of the floor, yet a complete metric traversability map of the surroundings would better serve applications such as indoor navigation.

By Subhransu S. Bhattacharjee, Dylan Campbell, Rahul Shome

robotics benchmarks

arXiv AI

Jun 30

Resonant Brane Splatting for Arbitrary-Scale Super-Resolution

arXiv:2606. 29453v1 Announce Type: cross Abstract: Arbitrary-Scale Super-Resolution (ASR) reconstructs images at continuous magnification factors.

By Giulio Federico, Giuseppe Amato, Claudio Gennaro, Fabio Carrara, Marco Di Benedetto

multimodal benchmarks

arXiv Machine Learning

Jun 30

Attention Enhanced Entity Recommendation for Intelligent Monitoring in Cloud Systems

arXiv:2510. 20640v2 Announce Type: replace Abstract: In this paper, we present DiRecGNN, an attention-enhanced entity recommendation framework for monitoring cloud services at Microsoft.

By Fiza Husain, Anson Bastos, Anjaly Parayil, Ayush Choure, Chetan Bansal, Rujia Wang, Saravan Rajmohan

llms efficiency benchmarks

arXiv AI

Jun 30

An AI agent for treatment reasoning over a biomedical tool universe

arXiv:2606. 28692v1 Announce Type: new Abstract: Treatment reasoning underpins every therapeutic decision, integrating disease context, comorbidities, medications, contraindications, and evolving biomedical knowledge to select an appropriate therapy.

By Shanghua Gao, Ayush Noori, Richard Zhu, Curtis Ginder, Zhenglun Kong, Xiaorui Su, Justin Kauffman, Benjamin S. Glicksberg, Joshua Lampert, Ankit Sakhuja, Ashwin Sawant, ATHENA-R1 Evaluation Consortium, David A. Clifton, Noa Dagan, Ran Balicer, Marinka Zitnik

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing

arXiv:2603. 03143v2 Announce Type: replace-cross Abstract: Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm.

By Jiyuan Wang, Chunyu Lin, Lei Sun, Zhi Cao, Yuyang Yin, Lang Nie, Zhenlong Yuan, Xiangxiang Chu, Yunchao Wei, Kang Liao, Guosheng Lin

diffusion computer-vision reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models

arXiv:2606. 29600v1 Announce Type: cross Abstract: A faithful 3D world representation should account for layered geometry, where a single camera ray may contain multiple visible and geometrically valid surfaces.

By Xiaohao Xu, Feng Xue, Xiang Li, Haowei Li, Shusheng Yang, Tianyi Zhang, Matthew Johnson-Roberson, Xiaonan Huang

llms computer-vision benchmarks

arXiv AI

Jun 30

Multi-Agent Routing as Set-Valued Prediction: A WildChat Benchmark and Cost-Aware Evaluation

arXiv:2606. 28925v1 Announce Type: cross Abstract: Tool and agent routing from natural-language prompts is naturally a set-valued prediction problem: a single query may require multiple agents, while over-selection increases execution cost.

By Ananto Nayan Bala, Faisal Muhammad Shah

llms agents fine-tuning benchmarks

arXiv AI

Jun 30

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).

By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 30

RA-QA: A Benchmarking System for Respiratory Audio Question Answering Under Real-World Heterogeneity

arXiv:2602. 18452v3 Announce Type: replace-cross Abstract: As conversational multimodal AI tools are increasingly adopted to process patient data for health assessment, robust benchmarks are needed to measure progress and expose failure modes under realistic conditions.

By Gaia A. Bertolino, Yuwei Zhang, Tong Xia, Domenico Talia, Cecilia Mascolo

llms nlp multimodal benchmarks

arXiv Machine Learning

Jun 30

Spectral Embedding via Chebyshev Bases for Robust DeepONet Approximation

arXiv:2512. 09165v2 Announce Type: replace Abstract: Deep Operator Networks (DeepONets) have emerged as a powerful framework for data-driven operator learning, providing flexible surrogates for nonlinear mappings arising in partial differential equations (PDEs).

By Muhammad Abid, Omer San

rag diffusion benchmarks safety

arXiv Machine Learning

Jun 30

Exploring Differences Between Tabular Enterprise Data and Public Benchmarks

arXiv:2606. 30452v1 Announce Type: new Abstract: Tabular data dominate the landscape of data science, increasingly attracting innovative machine learning models and tailored benchmarks.

By Myung Jun Kim, Maximilian Schambach, Frank Essenberger, Andre Sres, Johannes H\"ohne

benchmarks

arXiv AI

Jun 30

Temporal Feature Extractors in EEG Foundation Models: A Controlled Comparison Including a Pretrained Time-Series Model

arXiv:2606. 30104v1 Announce Type: new Abstract: Electroencephalography (EEG) foundation models aim to learn generalizable representations from large-scale brain recordings.

By Ay\c{s}e Bet\"ul Y\"uce, Chris Joey Leffler, Sarun Varghese, Myra Spiliopoulou, Sebastian Stober

benchmarks

arXiv Machine Learning

Jun 30

STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy

arXiv:2606. 29592v1 Announce Type: new Abstract: A central premise of autonomous scientific imaging is that smarter navigation, whether Bayesian, RL-based, or otherwise adaptive, is the principal lever for sample-efficient acquisition.

By Can Polat, Erchin Serpedin, Mustafa Kurban, Hasan Kurban

llms agents reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jun 30

GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks

arXiv:2606. 29773v1 Announce Type: new Abstract: Graphs are widely used to model relational systems, with applications in domains such as social networks, finance, and biomedicine.

By Haoxin Sun, Yiqing Lin, Yajun Huang, Chenhui Dong, Mingjun Li, Zhongzhi Zhang

llms diffusion fine-tuning benchmarks safety

arXiv AI

Jun 30

ReactiveBFM: Reactive Closed-Loop Motion Planning Towards Universal Humanoid Whole-Body Control

arXiv:2606. 30362v1 Announce Type: cross Abstract: While current Behavior Foundation Models (BFMs) provide robust control priors for humanoids, they only execute pre-defined reference motions.

By Xiao Chen, Weishuai Zeng, Xiaojie Niu, Zirui Wang, Jianan Li, Huayi Wang, Furui Xu, Jiahe Chen, Weixiang Zhong, Lihe Ding, Kailin Li, Jiangmiao Pang, Tai Wang, Tianfan Xue, Jingbo Wang

rag benchmarks safety

arXiv AI

Jun 30

Dual-Flow Reinforcement Learning with State-Aware Exploration

arXiv:2606. 29820v1 Announce Type: cross Abstract: In complex continuous-control reinforcement learning tasks, multimodal optimal actions often coincide with uncertain, multimodal return distributions, making reliable value estimation and multimodal exploration challenging.

By Qijun Li, Zheng Fu, Qi Song, Yifei He, Weitao Zhou, Kun Jiang, Diange Yang

diffusion reinforcement-learning multimodal benchmarks

arXiv AI

Jun 30

Beyond IID: How General Are Tabular Foundation Models, Really?

arXiv:2606. 30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.

By Lennart Purucker, Andrej Tschalzev, Nick Erickson, Gioia Blayer, David Holzm\"uller, Alan Arazi, Alexander Pfefferle, Mustafa Tajjar, Ga\"el Varoquaux, Frank Hutter

benchmarks

arXiv AI

Jun 30

LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

arXiv:2606. 28445v1 Announce Type: cross Abstract: Early detection of dementia enables timely intervention, and reflecting cognitive impairment, spontaneous speech offers a non-invasive screening modality.

By Jonghyeon Park, Olivier Jiyoun Jung, Myungwoo Oh

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 30

Field Order Should Not Matter: Permutation-Invariant Embedding Model Fine-Tuning for Structured Metadata Retrieval

arXiv:2606. 30473v1 Announce Type: cross Abstract: We study retrieval over catalogs of structured metadata, where each record is a small schema whose fields answer different kinds of query.

By Aivin V. Solatorio, Olivier Dupriez, Rafael Macalaba

llms rag agents fine-tuning benchmarks

arXiv AI

Jun 30

Collective cooperation without individual fidelity in LLM agents

arXiv:2606. 30454v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making.

By Henrique Ferraz de Arruda, Carlos Gracia L\'azaro, Alberto Aleta, Yamir Moreno

llms agents benchmarks

Get the daily briefing