Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,561 stories · RSS feed

arXiv AI

Jun 16

Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes

arXiv:2606. 15307v1 Announce Type: cross Abstract: Hateful and propagandistic memes exploit the interplay between images and text to convey harmful intent that neither modality reveals alone.

By Mohamed Bayan Kmainasi, Mucahid Kutlu, Ali Ezzat Shahroor, Abul Hasnat, Firoj Alam

llms reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Jun 16

RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs

arXiv:2602. 05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance.

By Youngcheon You, Banseok Lee, Minseop Choi, Seonyoung Kim, Hyochan Chong, Changdong Kim, Youngmin Kim, Dongkyu Kim

llms efficiency benchmarks

arXiv AI

Jun 16

Entropy-Gated Latent Recursion

arXiv:2606. 16620v1 Announce Type: cross Abstract: Inference-time scaling has become the dominant lever for improving language-model reasoning, but existing methods derive rollout diversity from a single source: stochastic token-level sampling.

By Soham Bhattacharjee, Dushyant Singh Chauhan, Salem Lahlou, Martin Takac, Nils Lukas

reinforcement-learning benchmarks

arXiv AI

Jun 16

Distilling Drifting Transformers with Representation Autoencoders

arXiv:2606. 15553v1 Announce Type: cross Abstract: Representation Autoencoders (RAEs) have improved diffusion and flow models by semantically richer latent space owing to the strongly label-wise clustered DINO features in the pretrained encoders.

By Jiawei Zhang, Mengfei Xia, Gen Li, Yuantao Gu

llms diffusion efficiency benchmarks

arXiv AI

Jun 16

Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling

arXiv:2605. 27023v2 Announce Type: replace Abstract: Knowledge graphs (KGs) have become the core backbone of numerous downstream tasks such as question answering and recommender systems.

By Yinan Liu, Wenjin Xu, Zhiyuan Zha, Xiaochun Yang, Bin Wang

rag nlp benchmarks

arXiv Machine Learning

Jun 16

Elastic ODYN: Differentiable Optimization for Infeasible Control and Learning in Robotics

arXiv:2606. 16564v1 Announce Type: cross Abstract: Robotic systems routinely encounter conflicting objectives, modeling errors, and degenerate contact conditions that render quadratic programs (QPs) infeasible.

By Aristotelis Papatheodorou, Jose Rojas, Ioannis Havoutis, Carlos Mastalli

robotics benchmarks

arXiv AI

Jun 16

RecourseBench: A Modular Framework for Reproducible Algorithmic Recourse Evaluation

arXiv:2606. 16113v1 Announce Type: new Abstract: Algorithmic recourse methods provide counterfactual explanations that inform individuals of the actions required to overturn an unfavorable model decision.

By Zahra Khotanlou, Hashir Ahmed, Chenghao Tan, Ahmed Abdelaal, Amir-Hossein Karimi

benchmarks

Hugging Face Trending Papers

Jun 15

FusionRS: A Large-Scale RGB-Infrared Remote Sensing Dataset for Dual-Modal Vision-Language Foundation Models

Remote sensing vision-language models have advanced Earth observation understanding, but most existing work remains centered on RGB imagery, leaving the complementary information in infrared data underexplored. Infrared images provide distinctive cues, including thermal intensity structures, object boundaries, and illumination-invariant scene features, which can enrich visual-language learning beyond conventional RGB observations.

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 15

Curvature-Informed Potential Energy Surface for Protein-Ligand Binding Affinity Prediction

arXiv:2606. 14217v1 Announce Type: new Abstract: Accurate prediction of protein-ligand binding affinity is essential for structure-based drug discovery.

By Peng-Fei Sun, Chuan-Xian Ren, Hong Yan

benchmarks safety

arXiv AI

Jun 15

CineOrchestra: Unified Entity-Centric Conditioning for Cinematic Video Generation

arXiv:2606. 13768v1 Announce Type: cross Abstract: Cinematic video depicts multiple subjects acting or interacting at specific moments, captured with deliberate camera movement, and stitched together by shot transitions.

By Sharath Girish, Tsai-Shien Chen, Zhikang Dong, Mukesh Singhal, Hao Chen, Sergey Tulyakov, Aliaksandr Siarohin

rag diffusion benchmarks

arXiv Machine Learning

Jun 15

A Longitudinal Attribute-Conditioned Neural Network for Modeling Health-State Transition Probabilities in Temporally Irregular Data: The LANTERN Framework

arXiv:2606. 13880v1 Announce Type: new Abstract: Accurate estimation of long-term care transition probabilities is central to disability insurance pricing, reserving, and solvency assessment.

By Bright Kwaku Manu, Beckett Sterner, Petar Jevtic

benchmarks

arXiv AI

Jun 15

Clay-CNN Hybrids: Leveraging Geo-Foundational Models as Auxiliary Context for Landslide Detection

arXiv:2606. 14081v1 Announce Type: cross Abstract: Rapid post-event landslide mapping is essential for disaster response but remains difficult to automate due to extreme class imbalance.

By Huong Binh Vu

computer-vision fine-tuning benchmarks

arXiv Machine Learning

Jun 15

Zero-shot generalization of transformer neural operators to larger domains

arXiv:2606. 14597v1 Announce Type: new Abstract: Transformer-based neural operators have shown remarkable performance for approximating solution operators of partial differential equations on complex geometries.

By Armand de Villeroch\'e, Sibo Cheng, Vincent Le Guen, Marc Bocquet, Rem-Sophia Mouradi, Patrick Armand, Alban Farchi, Patrick Massin

llms rag benchmarks safety

arXiv Machine Learning

Jun 15

Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models

arXiv:2606. 14222v1 Announce Type: new Abstract: The rapid evolution of Time Series Foundation Models (TSFMs) has advanced zero-shot forecasting across diverse domains.

By Xilin Dai, Yiding Liu, Hongjie Xia, Yifan Hu, Zewei Dong, Jiang-Ming Yang, Qiang Xu

llms fine-tuning benchmarks

arXiv AI

Jun 15

Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs

arXiv:2606. 13815v1 Announce Type: new Abstract: Strategic reasoning under uncertainty underpins consequential decisions in negotiation, finance, and policy, but prevailing game-play benchmarks collapse heterogeneous reasoning dimensions into a single scalar, leaving the capability structure of frontier LLMs unexamined.

By Pratham Singla, Shivank Garg, Vihan Singh

llms benchmarks

arXiv AI

Jun 15

Numbers Already Carry Their Own Embeddings

arXiv:2606. 14108v1 Announce Type: cross Abstract: We introduce Adelic operation-preserved embeddings (AOE), a training-free representation that captures both a number's real value and its modular (p-adic) signatures.

By Suhyun Bae, Donghun Lee

rag benchmarks

arXiv AI

Jun 15

A Deep Reinforcement Learning (DRL)-Based Transformer Method for Solving the Open Shop Scheduling Problem

arXiv:2606. 13682v1 Announce Type: new Abstract: The open shop scheduling problem (OSSP) arises in many industrial and service settings but remains computationally challenging as the number of jobs and machines increases.

By Faezeh Ardali, Mwembezi A. Nyelele, Gerald M. Knapp

llms reinforcement-learning benchmarks

arXiv AI

Jun 15

Fragile Knowledge, Robust Instruction-Following: The Width Pruning Dichotomy in Llama-3.2

arXiv:2512. 22671v3 Announce Type: replace-cross Abstract: Structured width pruning of GLU-MLP layers in Llama-3.

By Pere Martra

llms efficiency benchmarks

arXiv Machine Learning

Jun 15

When to Write and When to Suppress: Route-Specialized Dual Adapters for Memory-Assisted Knowledge Editing

arXiv:2606. 14668v1 Announce Type: new Abstract: Knowledge editing systems must update selected facts while preserving nearby but irrelevant behavior.

By Yining Huang

llms rag fine-tuning benchmarks

arXiv Machine Learning

Jun 15

CANN-EUCLID: unsupervised constitutive artificial neural network model discovery from full-field data

arXiv:2606. 14565v1 Announce Type: cross Abstract: Constitutive artificial neural networks (CANNs) provide interpretable material model discovery, but have so far been used in stress-supervised settings based on apparent stress-strain data from homogeneous tests.

By Benjamin Alheit, Siddhant Kumar, Mathias Peirlinck

efficiency benchmarks

Get the daily briefing