Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,620 stories · RSS feed

arXiv Machine Learning

Sep 10

GraphNOSE: A Graph Transformer in Olfaction

arXiv:2609.05694v1 Announce Type: new Abstract: Predicting olfactory qualities from molecular structure is an open problem in chemoinformatics. Although linear models can link molecular features to o...

By Mrityunjay Sharma, Sarabeshwar Balaji, Valentina Parma, Ritesh Kumar

llms rag benchmarks

arXiv Machine Learning

Sep 10

Budgeted Task-Aware Acquisition of Dynamic Networks

arXiv:2609.05862v1 Announce Type: new Abstract: Learning on dynamic graphs is difficult when changes in the underlying network are only partially observed. Acquiring current graph information incurs...

By Zihe Zhou

benchmarks

arXiv Machine Learning

Sep 10

The Oversight Gap: What LLM Safety Monitors Miss, and Why It Is Not Capability

arXiv:2609.07162v1 Announce Type: new Abstract: Several properties safety monitors are asked to certify, among them cross-tenant noninterference, sandbagging and evaluation awareness, are 2-safety hy...

By Xin Xu

llms benchmarks safety

arXiv Machine Learning

Sep 10

CLUES-WEASEL: No additional clues required to choose your time series clustering algorithm

arXiv:2609.07606v1 Announce Type: new Abstract: Time series data is very common in many real-world applications and in numerous domains, with increasing interest for automated information extraction...

By Johann Faouzi

benchmarks

arXiv Machine Learning

Sep 10

Physics-Informed Deep Learning for False Ventricular Tachycardia Alarm Reduction in the ICU

arXiv:2609.08992v1 Announce Type: new Abstract: False ventricular tachycardia (VT) alarms are a leading contributor to alarm fatigue in intensive care units. We propose a deep learning framework comb...

By Athanasios Papastathopoulos-Katsaros, Alexandra Stavrianidi, Zhandong Liu

benchmarks

arXiv Machine Learning

Sep 10

Srijika: OpenType-Layout-Reusing Font Restyling for Nine Indic Scripts

arXiv:2609.05661v1 Announce Type: cross Abstract: We present Srijika, a system for producing installable OpenType fonts for nine Brahmic scripts: Devanagari, Tamil, Bengali, Telugu, Kannada, Malayala...

By Anil Pai

rag diffusion benchmarks

arXiv Machine Learning

Sep 10

Novel hybrid protein scaffold gap filling using weighted machine learning ensemble, beam search, and mass-constrained reranking

arXiv:2609.05436v1 Announce Type: cross Abstract: Protein scaffold gap filling is an important computational task in protein sequence reconstruction, where missing amino acid regions must be inferred...

By Tahmid Enam Shrestha, Md. Manzurul Hasan, Md. Rafiqul Islam

rag benchmarks

arXiv Machine Learning

Sep 10

Flexible Spectral-Normalized Neural Gaussian Process for Dynamic Aperture Prediction

arXiv:2609.08620v1 Announce Type: cross Abstract: We address the challenge of scalable uncertainty quantification in large-scale scientific applications, where complex state-of-the-art machine learni...

By Yousra El-Bachir, Frederik Van der Veken, Davide di Croce, Carlo Emilio Montanari, Massimo Giovannozzi, Ekaterina Krymova, Tatiana Pieloni

benchmarks

arXiv Machine Learning

Sep 10

A Review of the Long Horizon Forecasting Problem in Time Series Analysis

arXiv:2506.12809v2 Announce Type: replace Abstract: The long horizon forecasting (LHF) problem has come up in the time series literature for over the last 35 years or so. This review covers aspects o...

By Hans Krupakar, Kandappan V A

efficiency benchmarks safety

arXiv AI

Sep 10

Agentic BAIM-LLM Evaluation (ABLE): Benchmarking LLM Use of Protein Design Tools

arXiv:2609.05818v1 Announce Type: new Abstract: We introduce ABLE, a benchmark for evaluating LLM agents' ability to use biological AI models (BAIMs), such as ProteinMPNN and AlphaFold3, in dual-use...

By Bryce Cai, Geetha Jeyapragasan, Samira Nedungadi, Jake Yukich, Seth Donoughe

llms agents benchmarks

arXiv Machine Learning

Sep 10

Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problems

arXiv:2603.20589v2 Announce Type: replace Abstract: Generating data from discrete distributions is important for a number of application domains including text, tabular data, and genomic data. Severa...

By Alankrita Bhatt, Mukur Gupta, Germain Kolossov, Andrea Montanari

benchmarks

arXiv Machine Learning

Sep 10

Subspace Optimization for Backpropagation-Free Continual Test-Time Adaptation

arXiv:2603.28678v2 Announce Type: replace Abstract: We introduce PACE, a backpropagation-free continual test-time adaptation system that directly optimizes the affine parameters of normalization laye...

By Damian S\'ojka, Sebastian Cygert, Marc Masana

benchmarks

arXiv Computation and Language

Sep 10

VLX-VR: An Agentic-Aware Video Reasoning Model

arXiv:2609.09985v1 Announce Type: new Abstract: Real-world video understanding requires integrating visual, audio, textual, and temporal evidence distributed across a video. Yet many pipelines use a...

By Sheng Li, Peng Liu, Qianqian Zhang, Tiancheng Zhao

agents reinforcement-learning multimodal benchmarks

arXiv Computer Vision

Sep 10

Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases

arXiv:2609.09462v1 Announce Type: new Abstract: Prompt learning adapts CLIP to downstream recognition by replacing hand-written templates with learned continuous context vectors, which in Context Opt...

By Tanvir Muntakim Tonoy, Sajjad Ghiasvand, Mahnoosh Alizadeh, Ramtin Pedarsani

llms rag multimodal benchmarks

arXiv Computer Vision

Sep 10

IAE-VTG: Interaction-Aligned Action-Entity Video Temporal Grounding

arXiv:2609.09736v1 Announce Type: new Abstract: Video Temporal Grounding (VTG) localizes the video segment that matches a natural-language query. Many queries describe an action performed by a partic...

By Shiwen Zhao, Qi Zhang, Sezer Karaoglu, Theo Gevers, Martin R. Oswald

benchmarks

arXiv Computer Vision

Sep 10

Can We Trust Video Hallucination Detectors? VidHalLoc for Evaluating the Evaluators

arXiv:2609.09895v1 Announce Type: new Abstract: Video-language models and video agents can produce hallucinations that conflict with spatiotemporal evidence. Existing benchmarks mainly evaluate model...

By Xinyu Chen, Adnan Mahmood, Mark Dras

llms agents nlp benchmarks safety

arXiv Computer Vision

Sep 10

Anatomy-Grounded Weakly Supervised Prompt Tuning for Chest X-ray Latent Diffusion Models

arXiv:2506.10633v2 Announce Type: replace Abstract: Latent Diffusion Models have shown remarkable results in text-guided image synthesis in recent years. In the domain of natural (RGB) images, recent...

By Konstantinos Vilouras, Ilias Stogiannidis, Junyu Yan, Alison Q. O'Neil, Sotirios A. Tsaftaris

diffusion fine-tuning multimodal benchmarks safety

arXiv Computer Vision

Sep 10

Multi-dimensional Preference Alignment by Conditioning Reward Itself

arXiv:2512.10237v2 Announce Type: replace Abstract: Reinforcement Learning from Human Feedback has emerged as a standard for aligning diffusion models. However, we identify a fundamental limitation i...

By Jiho Jang, Jinyoung Kim, Kyungjune Baek, Nojun Kwak

diffusion reinforcement-learning benchmarks safety

arXiv Computer Vision

Sep 10

Towards Characterizing Scientific Image Utility and Upgradability

arXiv:2606.03401v2 Announce Type: replace Abstract: Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated conten...

By WenZhe Li, Qihang Yan, Liang Chen, Junying Wang, Farong Wen, Yijin Guo, Chunyi Li, Zicheng Zhang, Guangtao Zhai

llms multimodal benchmarks

arXiv Computer Vision

Sep 10

Predicting upcoming visual features during eye movements yields scene representations aligned with human visual cortex

arXiv:2511.12715v2 Announce Type: replace-cross Abstract: Natural scenes are complex arrangements of objects, surfaces, and backgrounds. For the brain's visual system to effectively operate, it needs...

By Sushrut Thorat, Adrien Doerig, Alexander Kroner, Carmen Amme, Tim C. Kietzmann

rag benchmarks

Get the daily briefing