Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,507 stories · RSS feed

arXiv Machine Learning

Jun 2

B-GRTO: Bootstrapped Group Relative Tool Optimization for Referring Segmentation

arXiv:2605. 23500v2 Announce Type: replace-cross Abstract: Segmentation is a fundamental task in computer vision, underpinning pixel-level scene understanding and serving as a cornerstone for applications ranging from autonomous perception to medical image analysis.

By Mario Markov (INSAIT, Sofia University "St. Kliment Ohridski"), Stefan Maria Ailuro (INSAIT, Sofia University "St. Kliment Ohridski"), Mohammad Mahdi (INSAIT, Sofia University "St. Kliment Ohridski"), Luc Van Gool (INSAIT, Sofia University "St. Kliment Ohridski"), Danda Pani Paudel (INSAIT, Sofia University "St. Kliment Ohridski")

llms agents computer-vision reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jun 2

From Moments to Models: Graphon-Mixture Learning for Mixup and Contrastive Learning

arXiv:2510. 03690v4 Announce Type: replace Abstract: Real-world graph datasets often arise from mixtures of populations, where graphs are generated by multiple distinct underlying distributions.

By Ali Azizpour, Reza Ramezanpour, Santiago Segarra

benchmarks

arXiv AI

Jun 2

OPD+: Rethinking the Advantage Design for On-Policy Distillation

arXiv:2606. 01039v1 Announce Type: cross Abstract: On-policy distillation (OPD) is a widely used technique to transfer capabilities from capable teacher language models to the base student models, and can be formulated in a reinforcement learning style objective using student generated rollouts.

By Hanyang Zhao, Haoxian Chen, Han Lin, Genta Indra Winata, David Yao, Wenpin Tang

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 2

Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics

arXiv:2602. 24201v2 Announce Type: replace Abstract: Estimating density ratios between pairs of intractable data distributions is a core problem in probabilistic modeling, enabling principled comparisons of sample likelihoods under different data-generating processes across conditions.

By Egor Antipov, Alessandro Palma, Lorenzo Consoli, Stephan G\"unnemann, Andrea Dittadi, Fabian J. Theis

diffusion benchmarks

arXiv Machine Learning

Jun 2

TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning

arXiv:2606. 01591v1 Announce Type: cross Abstract: The TimeLogic Challenge evaluates formal temporal-logic reasoning over video - 16 operators (before, after, until, since, always, co-occur, ordering, ...

By Ali Alavi

llms nlp multimodal benchmarks

arXiv Machine Learning

Jun 2

ChWDTA: Channel-wise Wavelet-Domain Transformer Attention and Entropy Modeling for Learned Image Compression

arXiv:2606. 00111v1 Announce Type: cross Abstract: State-of-the-art learned image compression (LIC) schemes are increasingly based on hybrid CNN-transformer architectures.

By Haisheng Fu, Runyu Yang, Feng Ding, Siyu Zhu, Jie Liang, Xiaoxiao Li, Zhenman Fang, Jingning Han

llms nlp benchmarks

arXiv AI

Jun 2

An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentation

arXiv:2606. 00987v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have shown strong visual understanding and language-guided grounding abilities, yet their capacity for multi-temporal visual reasoning remains underexplored.

By Bingyu Li, Da Zhang, Tao Huo, Zhiyuan Zhao, Junyu Gao, Xuelong Li

llms agents computer-vision fine-tuning multimodal benchmarks

arXiv AI

Jun 2

Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs

arXiv:2606. 01400v1 Announce Type: cross Abstract: Evaluating large language models (LLMs) across comprehensive benchmarks is expensive and time-consuming.

By Denica Kjorvezir, Marko Djukanovi\'c, Ana Gjorgjevikj, Gjorgjina Cenikj, Tome Eftimov

llms rag benchmarks

arXiv AI

Jun 2

Experiments in Agentic AI for Science

arXiv:2605. 26305v2 Announce Type: replace Abstract: This paper details two novel frameworks for developing autonomous, agentic AI in scientific workflows.

By Judy Fox, Geoffrey Fox

llms rag agents benchmarks

arXiv AI

Jun 2

When Softmax Fails at the Top: Extreme Value Corrections for InfoNCE

arXiv:2606. 00262v1 Announce Type: cross Abstract: InfoNCE is the standard contrastive learning objective, but its softmax form is not only a computational convenience: it also encodes a statistical assumption about how the top-scoring example is selected.

By Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng

rag benchmarks

arXiv AI

Jun 2

DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle

arXiv:2307. 06647v4 Announce Type: replace-cross Abstract: We propose DeepIPCv2, an end-to-end autonomous driving framework that integrates LiDAR-based environmental perception with command-specific control learning.

By Oskar Natan, Jun Miura

agents computer-vision robotics benchmarks

arXiv AI

Jun 2

Identifying High-Confidence Social Biases in LLMs for Trustworthy Conversational Tutoring Agents

arXiv:2606. 01584v1 Announce Type: cross Abstract: Conversational tutoring agents have been shown to improve learning engagement and student outcomes, and large language models (LLMs) are increasingly used in these systems to provide scalable, personalized feedback.

By Aitor Arronte Alvarez, Naiyi Xie Fincham

llms agents benchmarks safety

arXiv AI

Jun 2

Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying

arXiv:2606. 00151v1 Announce Type: cross Abstract: In reinforcement learning (RL), agents benefit from exploration only because they repeatedly encounter similar states: trying different actions can improve performance or reduce uncertainty; without such retries, a greedy policy is optimal.

By Soichiro Nishimori, Paavo Parmas, Sotetsu Koyamada, Tadashi Kozuno, Toshinori Kitamura, Shin Ishii, Yutaka Matsuo

agents reinforcement-learning benchmarks

arXiv AI

Jun 2

JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

arXiv:2606. 01703v1 Announce Type: cross Abstract: We address the challenge of generating high-fidelity, long-form soundtracks that remain coherent across scene transitions.

By Jiashuo Yu, Yao Yao, Boyu Chen, Alex Wang

llms agents multimodal benchmarks safety

arXiv Machine Learning

Jun 2

Are Large Reasoning Models Interruptible?

arXiv:2510. 11713v4 Announce Type: replace-cross Abstract: Real-world applications of Large Reasoning Models (LRMs) often require reasoning about changing prompts or environments.

By Tsung-Han Wu, Mihran Miroyan, David M. Chan, Trevor Darrell, Narges Norouzi, Joseph E. Gonzalez

benchmarks

arXiv Machine Learning

Jun 2

ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?

arXiv:2606. 01849v1 Announce Type: new Abstract: Differentially private (DP) text synthesis promises to unlock sensitive corpora for model training, but it remains unclear whether DP synthetic data transmits genuinely new knowledge and capabilities present only in those corpora.

By Peihan Liu, Lucas Rosenblatt, Weiwei Kong, Natalia Ponomareva, Gautam Kamath, Rachel Cummings, Roxana Geambasu, Yu Gan, Lillian Tsai, Alex Bie

benchmarks

arXiv Machine Learning

Jun 2

LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems

arXiv:2606. 01111v1 Announce Type: new Abstract: Modern industrial recommender systems rely on thousands of heterogeneous features -- ranging from low-dimensional scalars (e.

By Yihong Huang, Chen Chu, Fei Chen, Yu Lin, Ruiduan Li, Zhihao Li

rag efficiency benchmarks

arXiv Machine Learning

Jun 2

RADE: Random Add-Drop Edge as a Regularizer

arXiv:2606. 00757v1 Announce Type: new Abstract: Graph Neural Networks (GNNs) suffer from overfitting and over-squashing of long-range information.

By Danial Saber, Amirali Salehi-Abari

benchmarks safety

arXiv AI

Jun 2

StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement

arXiv:2606. 00267v1 Announce Type: cross Abstract: Video world models (WMs) have shown promise for policy evaluation and improvement by imagining realistic future observations conditioned on ego-robot actions.

By Junwon Seo, Sushant Veer, Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Marco Pavone, Andrea Bajcsy

llms agents diffusion robotics multimodal benchmarks

arXiv Machine Learning

Jun 2

Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking

arXiv:2602. 03912v4 Announce Type: replace Abstract: This paper investigates the performance of Echo State Networks (ESNs) for univariate forecasting of monthly and quarterly time series from the M4 Forecasting Competition dataset.

By Alexander H\"au{\ss}er

benchmarks

Get the daily briefing