Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Machine Learning

Oct 2

GraphMAS: A Systematic Benchmark of Multi-Agent Coordination for Graph Learning

arXiv:2609.39777v1 Announce Type: new Abstract: LLM-based multi-agent systems coordinate specialized reasoning through aggregation, interaction, and adaptive control, yet their potential for graph le...

By Jiayi Yang, Yifang Chen, Yuanfu Sun, Xinyan Ge, Qiaoyu Tan

llms agents benchmarks

arXiv Machine Learning

Oct 2

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

arXiv:2609.39810v1 Announce Type: new Abstract: Source-Free Universal Domain Adaptation (SF-UniDA) extends Universal Domain Adaptation by removing access to source data at adaptation time while still...

By Romain Mussard, Fannia Pacheco, Maxime Berar, Paul Honeine, Gilles Gasso

fine-tuning benchmarks

arXiv Machine Learning

Oct 2

RainAtlas: A Multi-Continental Dataset for Precipitation Downscaling

arXiv:2609.39833v1 Announce Type: new Abstract: Extreme rainfall events are increasing in intensity and frequency as climate change accelerates. While kilometer-scale precipitation forecasts are crit...

By Pierre-Louis Lemaire, Luca Schmidt, Wietze Suijker, Alex Hernandez-Garcia, David Rolnick

benchmarks

arXiv Machine Learning

Oct 2

Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning

arXiv:2609.39839v1 Announce Type: new Abstract: Class-Incremental Learning (CIL) aims to continuously learn new classes without forgetting previously acquired knowledge. Parameter-efficient fine-tuni...

By Hongwei Zhao (School of Computer Science,Engineering, Beihang University), Rui Liu (School of Computer Science,Engineering, Beihang University), Yansong Liu (School of Computer Science,Engineering, Beihang University)

fine-tuning benchmarks safety

arXiv Machine Learning

Oct 2

From DNA Design to DNA Slimming: Auditable Agentic Discovery of a Deletion-Only Designer

arXiv:2609.40143v1 Announce Type: new Abstract: Compact regulatory DNA can free up space in vector payloads, reduce synthesis and assay burden, and expose which sequence features drive predicted acti...

By Joel Shor

agents benchmarks

arXiv Machine Learning

Oct 2

Should I stay or should I show? Learning to selectively disclose information

arXiv:2609.39818v1 Announce Type: new Abstract: In many high-stakes settings, human decision-makers can acquire support information before making a decision. However, acquiring information is costly,...

By Carlotta Giacchetta, Alessando Bogani, Cesare Barbera, Giovanni De Toni, Michele Caprio, Andrea Pugnana, Andrea Passerini

benchmarks

arXiv Machine Learning

Oct 2

About the Influence of Workflow Topology on Task Intensity Prediction through Graph Learning

arXiv:2609.39481v1 Announce Type: cross Abstract: Efficient resource provisioning for large-scale workflows on cloud infrastructures is a critical performance engineering challenge. These workflows a...

By Max Otto, Haci Ismail Aslan, Joel Witzke, Jonathan Bader, Odej Kao

benchmarks

arXiv Machine Learning

Oct 2

Cybersecurity in Edge Computing: A Trust-Aware Federated Hybrid Intrusion Detection Framework

arXiv:2609.39584v1 Announce Type: cross Abstract: Edge computing has emerged as a critical computing paradigm in modern distributed systems by migrating data processing closer to end users and Intern...

By Zawad Yalmie Sazid, Robert Abbas

agents benchmarks safety

arXiv Machine Learning

Oct 2

PassGPT+: Leveraging Linguistic Priors for Password Modeling

arXiv:2609.39880v1 Announce Type: cross Abstract: Passwords remain the dominant online authentication mechanism, and understanding how humans choose them is essential for defensive strength estimatio...

By Rajneesh Anand, Neeraj Lakshmanan, Masoud Yari

llms diffusion nlp benchmarks

arXiv Machine Learning

Oct 2

HyDRA: A Hybrid Dual-Mode Network for Closed- and Open-Set RFFI with Optimized VMD

arXiv:2507.12133v2 Announce Type: replace Abstract: Device recognition is vital for security in wireless communication systems, particularly for applications like access control. Radio Frequency Fing...

By Hanwen Liu, Yuhe Huang, Yifeng Gong, Yanjie Zhai, Jiaxuan Lu

llms efficiency benchmarks

arXiv Machine Learning

Oct 2

Contrastive Representation Shaping for LLM Unlearning

arXiv:2601.22028v2 Announce Type: replace Abstract: Most LLM unlearning methods aim to approximate retrain-from-scratch behaviors with minimal distribution shift, often via alignment-style objectives...

By Haoran Tang, Rajiv Khanna

llms rag benchmarks safety

arXiv Machine Learning

Oct 2

MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models

arXiv:2602.03537v2 Announce Type: replace Abstract: Matryoshka Quantization (MatQuant), Any-Precision-LLM (AP) and AnyBCQ (AB) are recent quantization approaches showing that a single integer-quantiz...

By Maximilian Kleinegger, Elvir Crn\v{c}evi\'c, Dan Alistarh

llms efficiency benchmarks

arXiv Machine Learning

Oct 2

Rising Multi-Armed Bandits with Known Horizons

arXiv:2602.10727v3 Announce Type: replace Abstract: Rising Multi-Armed Bandits (RMABs) model sequential decision problems where each arm's expected reward improves with repeated pulls. In such proble...

By Seockbean Song, Chenyu Gan, Youngsik Yoon, Siwei Wang, Wei Chen, Jungseul Ok

reinforcement-learning benchmarks

arXiv Machine Learning

Oct 2

Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains

arXiv:2604.08454v2 Announce Type: replace Abstract: Large language models are increasingly deployed in high-stakes domains, where confident yet incorrect inferences may cause severe real-world harm,...

By Haokai Ma, Lee Yan Zhen, Gang Yang, Yunxiang Chen, Yunshan Ma, Tat-Seng Chua, Ee-Chien Chang

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Oct 2

Measuring Structure in Graph Benchmark Datasets Using Graph Invariants

arXiv:2605.06462v2 Announce Type: replace Abstract: Progress in graph learning is hindered by benchmark practices that conflate the contributions of node features and graph structure, making it hard...

By Richard von Moos, Mathieu Alain, Bastian Rieck

llms benchmarks

arXiv Machine Learning

Oct 2

Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoders

arXiv:2605.16339v2 Announce Type: replace Abstract: Preference learning in large language models relies on reward models as proxies for human judgment. However, these models frequently exhibit prefer...

By Shunchang Liu, Xin Chen, Belen Martin Urcelay, Francesco Croce

llms diffusion reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 2

Learning to Select Source-Traceable Evidence for Language-Model Prediction from Irregular Clinical Time Series

arXiv:2605.20292v2 Announce Type: replace Abstract: Numerical time-series models effectively process irregular electronic health record (EHR) trajectories, but do not expose which temporal patterns s...

By Kwanhyung Lee, Juhwan Choi, Jongheon Kim, Joohyung Lee, Hyeongwon Jang, Jeonguk Lee, Jisoo Jung, Eunho Yang

llms benchmarks

arXiv Machine Learning

Oct 2

EnsembleEGNN: Set-Based Graph Learning for Thermodynamic Ensembles of Cyclic Peptides

arXiv:2607.21561v2 Announce Type: replace Abstract: Molecular graph encoding often relies on a single, static structure, ignoring the thermodynamic ensemble of molecules that are present in solution....

By Aaron L. Feller, Kris Deibler, Maxim Secor

benchmarks

arXiv Machine Learning

Oct 2

Autonomous Fashion Outfit Composition via Unified Aesthetic Foresight Model

arXiv:2608.13888v2 Announce Type: replace Abstract: Fashion Outfit Composition (FOC) requires sequentially assembling fashion items into a stylistically cohesive ensemble. Existing works struggle to...

By Kaicheng Pang, Xingxing Zou, Ruohan Xu, Waikeung Wong

agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Oct 2

Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans

arXiv:2602.23934v2 Announce Type: replace-cross Abstract: This paper presents a novel autonomous robotic assembly framework for constructing stable structures without relying on predefined architectu...

By Jingwen Wang, Johannes Kirschner, Paul Rolland, Luis Salamanca, Stefana Parascho

agents reinforcement-learning robotics benchmarks

Get the daily briefing