Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv AI

1d ago

Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis

arXiv:2602.20207v4 Announce Type: replace-cross Abstract: Knowledge editing in Large Language Models (LLMs) aims to update the model's prediction for a specific query to a desired target while preser...

By Shrestha Datta, Hongfu Liu, Anshuman Chhabra

llms benchmarks

arXiv AI

1d ago

PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation

arXiv:2603.13275v4 Announce Type: replace-cross Abstract: Accurate prediction of surgical duration is pivotal for hospital resource management. Although recent supervised learning approaches, from ma...

By Wanyin Wu, Kanxue Li, Baosheng Yu, Haoyun Zhao, Yibing Zhan, Dapeng Tao, Hua Jin

llms rag fine-tuning benchmarks safety

arXiv AI

1d ago

TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots

arXiv:2603.18008v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for mental-health support; yet prevailing evaluation methods--fluency metrics, preference...

By Fangrui Huang, Souhad Chbeir, Arpandeep Khatua, Sheng Wang, Sijun Tan, Kenan Ye, Lily Bailey, Merryn Daniel, Ryan Louie, Sanmi Koyejo, Ehsan Adeli

llms reinforcement-learning benchmarks safety

arXiv AI

1d ago

MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels

arXiv:2603.19310v5 Announce Type: replace-cross Abstract: Reinforcement learning (RL) fine-tuning improves large language model (LLM) reasoning using reward signals on sampled rollouts. When reward l...

By Tianyang Luo, Tao Feng, Zhigang Hua, Yan Xie, Shuang Yang, Ge Liu, Jiaxuan You

llms nlp reinforcement-learning fine-tuning benchmarks

arXiv AI

1d ago

ReMoA: Recursive Modular Adaptation via Sparse Latent Refinement in Multimodal Diffusion Models

arXiv:2604.25299v2 Announce Type: replace-cross Abstract: Diffusion models have achieved success in high-fidelity data synthesis, yet their capacity for more complex, structured reasoning such as tex...

By Yuwei Sun, Yuxuan Yao, Hui Li, Weihao Yuan, Siyu Zhu

llms diffusion multimodal benchmarks

arXiv AI

1d ago

Structural Rationale Distillation via Reasoning Space Compression

arXiv:2605.07139v2 Announce Type: replace-cross Abstract: When distilling reasoning from large language models (LLMs) into smaller ones, teacher rationales for similar problems often vary wildly in s...

By Jialin Yang, Jiankun Wang, Jiajun Wu, Henry Leung, Jiayu Zhou, Steve Drew

llms efficiency benchmarks

arXiv AI

1d ago

KL for a KL: On-Policy Distillation with Control Variate Baseline

arXiv:2605.07865v2 Announce Type: replace-cross Abstract: On-Policy Distillation (OPD) has emerged as a dominant post-training paradigm for Large Language Models, especially in reasoning domains. How...

By Minjae Oh, Sangjun Song, Gyubin Choi, Yunho Choi, Yohan Jo

llms reinforcement-learning efficiency benchmarks

arXiv AI

1d ago

GraphIP-Bench: How Hard Is It to Steal a Graph Neural Network, and Can We Stop It?

arXiv:2605.12827v3 Announce Type: replace-cross Abstract: Graph neural networks (GNNs) deployed as cloud services can be *stolen* through *model-extraction attacks*, which train a surrogate from quer...

By Kaixiang Zhao, Bolin Shen, Yuyang Dai, Shayok Chakraborty, Yushun Dong

benchmarks

arXiv AI

1d ago

Exploiting Textual Semantics for Robust Cross-View Object Correspondence

arXiv:2611.02356v2 Announce Type: replace-cross Abstract: Cross-view object correspondence (CVOC) aims to establish associations of the same target between query and search videos captured from diffe...

By Bing Fan, Yunhe Feng, Yan Huang, Heng Fan

llms multimodal benchmarks

arXiv Machine Learning

1d ago

Matched but Not Better: An Equivalence-Tested Benchmark of Quantum Machine Learning on Six Clinical Datasets and the QML-Health-READY Reporting Framework

arXiv:2611.05442v1 Announce Type: new Abstract: Background: Quantum machine learning (QML) is promoted for clinical prediction, but comparisons with classical models are rarely paired or tested for e...

By Mohammed Siraj B, Gousia Habib, Ruban S

benchmarks

arXiv Machine Learning

1d ago

C-GRPO: Conformal Group Relative Policy Optimization

arXiv:2611.05591v1 Announce Type: new Abstract: Reasoning-oriented language model training increasingly relies on multiple on-policy rollouts, but standard Group Relative Policy Optimization (GRPO) u...

By Arya Fayyazi, Seyedarmin Azizi, Mehdi Kamal, Massoud Pedram

llms benchmarks

arXiv Machine Learning

1d ago

RIDGE: State-Conditioned Reward Blending for Behavioral Coverage in Deep RL Game Agents

arXiv:2611.05601v1 Announce Type: new Abstract: A deep RL agent trained on a fixed persona reward plays one way: an Explorer keeps exploring while starving, a Survivor hoards on an empty map. Since e...

By AL Muqshith Mohammed Shifan, Kevin Christopher Chua, Ali Neshati, Loutfouz Zaman, Cristiano Politowski

agents reinforcement-learning benchmarks

arXiv Machine Learning

1d ago

CoReFlow: Consistency-Relaxed Flow Maps for Probabilistic Traffic Forecasting

arXiv:2611.05715v1 Announce Type: new Abstract: Probabilistic traffic forecasting is essential for reliable Intelligent Transportation Systems (ITS), as it captures uncertainties in complex road netw...

By Pengnan Chi, Zhiguo Zhang, Xiaoliang Ma

diffusion efficiency multimodal benchmarks

arXiv Machine Learning

1d ago

Give the Drafter a Second Chance to Guess: Parallel Reconsideration for Speculative Decoding

arXiv:2611.05724v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference with a small drafter, whose per-guess acceptance rate determines the achievable speedup...

By Hoang Anh Duy Le, Huibing Dong, Alireza Khadem, Ziteng Sun, Xiang Song, Yizheng Huang, Soroush Radpour, Anshumali Shrivastava, Ye He

llms benchmarks

arXiv Machine Learning

1d ago

Self-Supervised Learning of Universal Latent Spaces for Operator Learning

arXiv:2611.05735v1 Announce Type: new Abstract: Neural operators fit their representation of functions to the task they are trained on, so every new task rebuilds a reduced space from labeled data. F...

By Sepehr Mousavi, Levi Lingsch, Siddhartha Mishra, Laura De Lorenzis

rag diffusion computer-vision nlp efficiency benchmarks

arXiv Machine Learning

1d ago

Decouple to Reuse: Fine-Grained Knowledge Augmentation for Time Series Foundation Models

arXiv:2611.05993v1 Announce Type: new Abstract: Time series forecasting is critical for demand prediction and resource management in modern Web services, yet applying pretrained Time Series Foundatio...

By Yuqian Wu, Qingxiang Liu, Yuxin Guo, Weifeng Su, Jian Li, Pengfei Xu, Xi Chen, Yutao Yue, Yuxuan Liang

rag benchmarks

arXiv Machine Learning

1d ago

How Far Can You Steer? Behavior-Capability Frontiers in Contrastive Activation Addition

arXiv:2611.06001v1 Announce Type: new Abstract: Activation steering can modify language-model behavior without parameter updates, but behavioral efficacy alone does not determine whether an intervent...

By Deepanshu Raj

benchmarks

arXiv Machine Learning

1d ago

TRACK: Multiple Instance Learning for Anti-Money Laundering in Transaction Networks

arXiv:2611.06035v1 Announce Type: new Abstract: Anti-money laundering (AML) investigations are organized around accounts, while the suspicious evidence for an account may reside in only a few of its...

By Seonkyu Lim, Inwoo Tae, Gwangui Hong, Jaehoon Lee, Jeongwhan Choi, Taehoon Kim, Min-Soo Kim, Yongjae Lee

benchmarks

arXiv Machine Learning

1d ago

Learning From Static Context in Federated Time Series Forecasting

arXiv:2611.06064v1 Announce Type: new Abstract: Time series forecasting is shifting from locally trained, per-series models to global models trained across many heterogeneous series, which requires l...

By Viktor Walter, Andreas Wagner, Fabian Stricker

llms benchmarks

arXiv Machine Learning

1d ago

Catastrophic Overfitting is Weight Drift the Training Loss Cannot See

arXiv:2611.06082v1 Announce Type: new Abstract: Fast adversarial training (FAT) makes robust training affordable by replacing the multi-step attack with a single-step one, but it suffers from catastr...

By Chao Pan

benchmarks safety

Get the daily briefing