Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,477 stories · RSS feed

arXiv AI

Jul 1

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

arXiv:2606. 31718v1 Announce Type: cross Abstract: Relation extraction (RE) for low-resource languages is typically constrained by the lack of annotated corpora.

By Dragos-Mitrut Vasile, Elena-Simona Apostol, Stefan-Adrian Toma, Adrian Paschke, Ciprian-Octavian Truica

llms fine-tuning benchmarks

arXiv AI

Jul 1

RoPoLL: Robust Panel of LLM Judges

arXiv:2606. 30931v1 Announce Type: new Abstract: The LLM Jury, a Panel of LLM Evaluators (PoLL) reporting consensus scores, has become a practical alternative to single-judge LLM evaluation, yet its statistical behavior remains poorly understood.

By Anish Acharya, Kris W Pan, Brian Verkhovsky

llms benchmarks safety

arXiv AI

Jul 1

A Reproducible Benchmark of Lightweight CNNs: Accuracy, Efficiency, and the Impact of Pretrained Initialization

arXiv:2505. 03303v3 Announce Type: replace-cross Abstract: Lightweight convolutional neural networks are often compared using results obtained with different training recipes, input settings, and pretrained checkpoints.

By Tasnim Shahriar

fine-tuning benchmarks

arXiv AI

Jul 1

On the Convergence of Self-Improving Online LLM Alignment

arXiv:2606. 31524v1 Announce Type: cross Abstract: The Self-Improving Alignment (SAIL) algorithm addresses distribution shift by reducing a bilevel formulation of the problem to an efficient, single-level method.

By Xudong Wu, Pangpang Liu, Vaneet Aggarwal, Jiayu Chen

llms benchmarks safety

arXiv Machine Learning

Jul 1

Structure-Regularized Interpretable TCR-Epitope Prediction

arXiv:2606. 30902v1 Announce Type: cross Abstract: T cell receptor (TCR)-epitope binding prediction is essential for understanding adaptive immunity and developing immunotherapies.

By Jiarui Li, Zixiang Yin, Yunbei Zhang, Janet Wang, Samuel J. Landry, Zhengming Ding, Ramgopal R. Mettu

llms rag benchmarks safety

arXiv AI

Jul 1

Gated Multi-Graph Fusion via Graph Attention Networks for Alzheimer's Disease Detection

arXiv:2606. 31186v1 Announce Type: cross Abstract: Spontaneous speech is a vital non-invasive biomarker for Alzheimer's Disease (AD), yet many systems overlook non-linear structural disruptions and clinical heterogeneity in pathological language.

By Jinyu Li, Xiao Wei, Bin Wen, Kai Li, Yuqin Lin, Xiaobao Wang, Longbiao Wang, Jianwu Dang

multimodal benchmarks

arXiv AI

Jul 1

Listening Between the Lines: Joint Learning of ASR Embeddings and LLM-Augmented Linguistics for Dementia Detection

arXiv:2606. 30675v1 Announce Type: cross Abstract: Early detection of dementia through speech analysis offers a non-invasive screening alternative, but capturing both acoustic and linguistic biomarkers remains challenging.

By Olivier Jiyoun Jung, Jonghyeon Park, Myungwoo Oh

llms rag multimodal benchmarks

arXiv AI

Jul 1

ASR-Agnostic Multimodal Spectrotemporal Modeling for Early Dementia Detection

arXiv:2606. 30646v1 Announce Type: cross Abstract: Speech recruits the same executive, attentional, and working memory processes underlying instrumental activities of daily living, or IADLs, providing a non-invasive proxy for cognitive assessment.

By Chukwuemeka Ugwu, Oluwafemi Richard Oyeleke

llms rag multimodal benchmarks

arXiv AI

Jul 1

Learning Structurally Consistent Representations for Multi-View Radar Semantic Segmentation

arXiv:2606. 31609v1 Announce Type: cross Abstract: Radar sensors provide reliable perception under adverse weather and lighting conditions, but their sparse, noisy, and weakly semantic measurements make dense semantic segmentation challenging.

By Ali Zia, Muhammad Umer Ramzan, Abdelwahed Khamis, Usman Ali, Abdul Rehman

computer-vision efficiency benchmarks safety

arXiv AI

Jul 1

Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided Supervision

arXiv:2606. 31800v1 Announce Type: new Abstract: Despite recent progress, the reasoning capabilities of large multimodal language models (MLLMs) remain fundamentally constrained by static supervision, where fixed prompts, rules, or reward models provide non-adaptive guidance throughout training.

By Xianda Zheng, Huan Gao, Meng-Fen Chiang, Michael Witbrock, Kaiqi Zhao, Shangyang Li

llms nlp multimodal benchmarks safety

arXiv AI

Jul 1

Cross-Modal Hierarchical Fusion for from Multi-Sensor Ground Observation

arXiv:2606. 30647v1 Announce Type: cross Abstract: Dense volumetric reconstruction of cloud microphysical fields from sparse ground-based instruments remains an open problem, largely because the available measurements are heterogeneous in both modality and spatial coverage.

By Xinze Zhang

multimodal benchmarks

arXiv Machine Learning

Jul 1

Partition-Guided Distance Saliency: Bridging Decision and Objective Spaces in Many-Objective Optimization

arXiv:2606. 30836v1 Announce Type: new Abstract: Explainability in Many-Objective Optimization (MaO) is currently hindered by the escalating complexity of the Pareto front, which renders the relationship between high-dimensional decision variables and objective outcomes increasingly opaque.

By Cl\'audio L\'ucio do Val Lopes, Fl\'avio Vin\'icius Cruzeiro Martins, Elizabeth Fialho Wanner

benchmarks safety

arXiv Machine Learning

Jul 1

T-QPM: Enabling Temporal Out-Of-Distribution Detection and Domain Generalization for Vision-Language Models in Open-World

arXiv:2603. 18481v2 Announce Type: replace-cross Abstract: Out-of-distribution (OOD) detection remains a critical challenge in open-world learning, where models must adapt to evolving data distributions.

By Aditi Naiknaware, Salimeh Sekeh

llms multimodal benchmarks

arXiv AI

Jul 1

AgentBound: Verifiable Behavioral Governance for Autonomous AI Agents

arXiv:2606. 30970v1 Announce Type: new Abstract: Autonomous AI agents increasingly perform consequential actions on behalf of human principals, including financial transactions, external communications, and enterprise workflows.

By Anuj Kaul, Qianlong Lan, Pranay Gupta

agents benchmarks safety

arXiv AI

Jul 1

Surrogate-Gated Generation and Foundation-Model Embeddings for Bayesian Materials Design

arXiv:2606. 28578v1 Announce Type: cross Abstract: Closed-loop materials discovery iterates between proposing candidate structures and evaluating their properties, and property evaluation dominates the cost.

By Sk Md Ahnaf Akif Alvi, Jan Janssen, Danny Perez, Douglas Allaire, Raymundo Arroyave

rag diffusion reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 1

Human-Agent Collaborative Paper-to-Page Crafting

arXiv:2510. 19600v2 Announce Type: replace-cross Abstract: In the quest for scientific progress, communicating research is as vital as the discovery itself.

By Qianli Ma, Siyu Wang, Yilin Chen, Yinhao Tang, Yixiang Yang, Chang Guo, Bingjie Gao, Zhening Xing, Yanan Sun, Zhipeng Zhang

agents multimodal benchmarks safety

arXiv AI

Jul 1

ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

arXiv:2606. 31054v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image.

By Zhiyuan Yao, Zheren Fu, Zhixiao Zheng, Jiajun Li, Yi Tu, Zhendong Mao

llms diffusion reinforcement-learning multimodal benchmarks safety

arXiv AI

Jul 1

Investigating Multi-Agent Deliberation in Law

arXiv:2606. 30906v1 Announce Type: new Abstract: Artificial Intelligence is increasingly applied to the field of law, and has the potential to increase access to justice.

By Cor Steging, Ludi van Leeuwen, Tadeusz Zbiegie\'n

llms agents benchmarks

arXiv AI

Jul 1

RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents

arXiv:2606. 22678v2 Announce Type: replace-cross Abstract: Agentic coding harnesses - such as Agent-Skills, Superpowers, and Agent-Rigor - are increasingly deployed to augment underlying LLMs for real-world software engineering tasks.

By Meher Bhaskar Madiraju, Meher Sai Preetam Madiraju

llms agents benchmarks

arXiv Machine Learning

Jul 1

Relational and Sequential Conformal Inference for Energy Time Series over Graphs via Foundation Models

arXiv:2606. 31804v1 Announce Type: new Abstract: Accurate energy demand forecasting is essential for the reliable operation and planning of modern sustainable energy systems.

By Keivan Faghih Niresi, Alice Cicirello, Olga Fink

llms benchmarks safety

Get the daily briefing