Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv AI

Oct 15

Learning Shortest Paths with Generative Flow Networks

arXiv:2603.01786v2 Announce Type: replace-cross Abstract: In this paper, we present a novel learning framework for finding shortest paths in graphs utilizing Generative Flow Networks (GFlowNets). Fir...

By Nikita Morozov, Ian Maksimov, Daniil Tiapkin, Sergey Samsonov

benchmarks

arXiv AI

Oct 15

Neural-Surrogate Co-Design of Aerodynamic Shape and Control

arXiv:2603.06760v2 Announce Type: replace-cross Abstract: Control co-design jointly optimizes a robot's physical design and its controller. Existing techniques are limited in the number of model para...

By Daniele Affinita, Mingda Xu, Beno\^it Valentin Gherardi, Pascal Fua

robotics benchmarks

arXiv AI

Oct 15

HybridStitch: Token-Adaptive Model Stitching for Efficient Diffusion Generation

arXiv:2603.07815v3 Announce Type: replace-cross Abstract: Iterative diffusion generation requires different amounts of model capacity at different token positions. Prior model combinations allocate c...

By Desen Sun, Jason Hon, Jintao Zhang, Sihang Liu

llms diffusion efficiency benchmarks

arXiv AI

Oct 15

DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay

arXiv:2603.16157v2 Announce Type: replace-cross Abstract: While Reinforcement Learning (RL) enhances Large Language Model reasoning, on-policy algorithms like GRPO are sample-inefficient as they disc...

By Long Li, Zhijian Zhou, Tianyi Wang, Weidi Xu, Zuming Huang, Wei Chu, Zhe Wang, Shirui Pan, Chao Qu, Yuan Qi

llms reinforcement-learning benchmarks data-engineering

arXiv AI

Oct 15

Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective

arXiv:2604.14025v2 Announce Type: replace-cross Abstract: Reconstructing 3D representations from 2D inputs is a fundamental task in computer vision and graphics, serving as a cornerstone for understa...

By Weijie Wang, Qihang Cao, Sensen Gao, Donny Y. Chen, Haofei Xu, Wenjing Bian, Songyou Peng, Tat-Jen Cham, Chuanxia Zheng, Andreas Geiger, Jianfei Cai, Jia-Wang Bian, Bohan Zhuang

computer-vision benchmarks

arXiv AI

Oct 15

Negative Advantages Is a Double-Edged Sword: Calibrating advantages in GRPO for Search Agents

arXiv:2604.18235v3 Announce Type: replace-cross Abstract: Search agents achieve strong question-answering performance through multi-turn interactions with search engines, with Group Relative Policy O...

By Jiayi Wu, Ruobing Xie, Zeqian Huang, Lei Jiang, Can Xu, Kangyang Luo, Bochen Lin, Ming Gao, Xiang Li

agents benchmarks

arXiv AI

Oct 15

MILP the Gap: An Automated Reasoning Approach to Condorcet's Paradox

arXiv:2604.19851v2 Announce Type: replace-cross Abstract: Computational social choice studies how conflicting individual preferences are aggregated into a collective decision, with applications to el...

By Itai Zilberstein, Ruben Martins, Ratip Emin Berker

benchmarks safety

arXiv AI

Oct 15

Call-Chain-Aware LLM-Based Test Generation for Java Projects

arXiv:2604.22046v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have recently shown strong potential for generating project-level unit tests. However, existing state-of-the-art...

By Guancheng Wang, Qinghua Xu, Lionel C. Briand, Zhaoqiang Guo, Kui Liu

llms benchmarks

arXiv AI

Oct 15

XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity

arXiv:2605.05662v2 Announce Type: replace-cross Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Mor...

By Dasol Choi, Eugenia Kim, Jaewon Noh, Sang Seo, Eunmi Kim, Myunggyo Oh, Yunjin Park, Brigitta Jesica Kartono, Josef Pichlmeier, Helena Berndt, Sai Krishna Mendu, Glenn Johannes Tungka, \"Ozlem G\"ok\c{c}e, Suresh Gehlot, Katherine Pratt, Amanda Minnich, Haon Park

llms multimodal benchmarks safety

arXiv AI

Oct 15

AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning

arXiv:2605.06149v2 Announce Type: replace-cross Abstract: The discount factor in reinforcement learning controls both the effective planning horizon and the strength of bootstrapping, yet most deep R...

By Yaomin Wang, Jianting Pan, Ran Tian, Xiaoyang Li, Yu Zhang, Hengle Qin, Tianshu YU

reinforcement-learning robotics benchmarks

arXiv AI

Oct 15

APEX: Assumption-free Projection-based Embedding eXamination Metric for Generative Distribution Evaluation

arXiv:2605.07786v4 Announce Type: replace-cross Abstract: As generative models achieve unprecedented visual quality, the gold standard for image evaluation remains traditional feature-distribution me...

By Caterina Gallegati, Monica Bianchini, Franco Scarselli, Vittorio Murino, Barbara Toniella Corradini

rag benchmarks safety

arXiv AI

Oct 15

Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models

arXiv:2605.13119v2 Announce Type: replace-cross Abstract: Vision-language-action (VLA) models are effective robot action executors, but they remain limited on long-horizon tasks due to the dual burde...

By Zixing Lei, Changxing Liu, Yichen Xiong, Minhao Xiong, Yuanzhuo Ding, Zhipeng Zhang, Weixin Li, Siheng Chen

llms agents robotics multimodal benchmarks

arXiv AI

Oct 15

CTI2TTP: Evidence-Grounded TTP Extraction from Cyber Threat Intelligence via Diverge-then-Converge Verification

arXiv:2605.25836v2 Announce Type: replace-cross Abstract: Extracting MITRE ATT&CK techniques from cyber threat intelligence (CTI) reports is an open-set, multi-label problem requiring both high recal...

By Yutong Cheng, Changze Li, Raihan Sultan Pasha Basuki, Qian Cui, Wei Ding, Peng Gao

llms benchmarks

arXiv AI

Oct 15

How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluation

arXiv:2606.00920v2 Announce Type: replace-cross Abstract: A benchmark score is a measurement whose uncertainty has two sources: an aleatoric component from stochastic decoding and an epistemic compon...

By Yongxi Zhou, Lai Yun Choi, Jiaxi Wen, Wenbo Ye, Junwei Yao

llms benchmarks

arXiv AI

Oct 15

HALLMARK: How to Diagnose LLM Citation Verifiers and When to Deploy Them

arXiv:2607.18360v2 Announce Type: replace-cross Abstract: GPTZero found 53 NeurIPS 2025 accepted papers with hallucinated citations, and venues now screen submissions with automated reference checker...

By Kayoon Kim, Wieland Brendel, Patrik Reizinger

llms agents benchmarks safety

arXiv AI

Oct 15

DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation

arXiv:2608.26019v2 Announce Type: replace-cross Abstract: On-policy self-distillation (OPSD) combines student generated trajectories with dense supervision from a privileged teacher of the same model...

By Yutong Chen, Guangfu Guo, Zhichao Xu, Kunpeng Liu

efficiency benchmarks

arXiv Machine Learning

Oct 15

Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

arXiv:2610.19121v1 Announce Type: new Abstract: Structured pruning reduces LLM inference costs by removing model components to produce smaller dense models. However, existing methods can cause reason...

By Woosang Lim, Haneul Yoo, Vincent-Daniel Yun, Sungjoo Yoo

llms efficiency benchmarks

arXiv Machine Learning

Oct 15

Token Utility for On-Policy Distillation via Local Trust-Region Optimization

arXiv:2610.18883v1 Announce Type: new Abstract: Token-selection methods for on-policy distillation (OPD) prioritize student-visited positions using diverse signals, including uncertainty, teacher-stu...

By Narayanan Sadagopan, Yilan Chen, Vijay Lingam

efficiency benchmarks

arXiv Machine Learning

Oct 15

Scaling Cheap Synthetic Pretraining Improves Indoor Radio-Map Prediction

arXiv:2610.19015v1 Announce Type: new Abstract: Real measurements are too limited to train an accurate indoor pathloss predictor that works across a wide range of buildings. Large-scale synthetic pre...

By Khoren Petrosyan, Artashes Mkrtchyan, Rafayel Mkrtchyan, Hrant Khachatrian, Theofanis P. Raptis

fine-tuning benchmarks

arXiv Machine Learning

Oct 15

What Information Buys in Active Inference

arXiv:2610.19046v1 Announce Type: new Abstract: Information-theoretic exploration is of primary interest in methodological studies of reinforcement learning (RL) and computational cognitive neuroscie...

By Melih Kandemir, Mustafa Mert \c{C}elikok

agents reinforcement-learning benchmarks

Get the daily briefing