Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv Machine Learning

Jun 4

Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction

arXiv:2602. 23214v2 Announce Type: replace-cross Abstract: Plug-and-Play diffusion prior (PnPDP) frameworks have emerged as a powerful paradigm for solving imaging inverse problems by treating pretrained generative models as modular priors.

By Chenhe Du, Xuanyu Tian, Qing Wu, Muyu Liu, Jingyi Yu, Hongjiang Wei, Yuyao Zhang

diffusion fine-tuning benchmarks safety

arXiv AI

Jun 4

HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series

arXiv:2605. 11130v4 Announce Type: replace-cross Abstract: Critical events in multivariate time series, from turbine failures to cardiac arrhythmias, demand accurate prediction, yet labeled data is scarce because such events are rare and costly to annotate.

By Jonas Petersen, Gian-Alessandro Lombardi, Riccardo Maggioni, Camilla Mazzoleni, Federico Martelli, Philipp Petersen

llms rag benchmarks

arXiv AI

Jun 4

Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text

arXiv:2605. 29076v2 Announce Type: replace-cross Abstract: LLMs have advanced text classification, yet existing paradigms face a trade-off: supervised (label only) fine-tuning is scalable but offers limited reasoning on complex text and lacks broader model transparency, while discrete prompt optimization offers human-readable instructions but struggles with performance and scalability.

By Tianyang Zhou, Wenbo Chen, Pierre Jinghong Liang, Leman Akoglu

llms nlp reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv AI

Jun 4

MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models

arXiv:2606. 04027v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising partially masked sequences under bidirectional context, exposing a safety surface distinct from autoregressive LLMs.

By Yingzi Ma, Zhengyue Zhao, Xiaogeng Liu, Minhui Xue, Yue Zhao, Chaowei Xiao

llms diffusion reinforcement-learning benchmarks safety

arXiv AI

Jun 4

CodegenBench: Can LLMs Write Efficient Code Across Architectures?

arXiv:2606. 04023v1 Announce Type: cross Abstract: While large language models (LLMs) have been extensively evaluated on code generation tasks for general-purpose programming and GPU-accelerated environments (e.

By Jie Li, Wenzhao Wu, Junqi Hu, Qinrui Zheng, Bowen Wu, Juepeng Zheng, Yutong Lu, Haohuan Fu

llms benchmarks

arXiv Machine Learning

Jun 4

Training-Free Lexical-Dense Fusion for Conversational-Memory Retrieval

arXiv:2606. 04194v1 Announce Type: new Abstract: Retrieving the few past turns that answer a new query across long multi-session histories is the retrieval bottleneck behind long-term conversational memory (LoCoMo, LongMemEval).

By Christian Lysenst{\o}en

rag benchmarks safety

arXiv AI

Jun 4

SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification

arXiv:2605. 13672v1 Announce Type: cross Abstract: Few-shot classification (FSC) is widely used for learning from limited labeled data, yet most evaluations implicitly assume that target concepts are independent of contextual cues.

By Giries Abu Ayoub, Morad Tukan, Loay Mualem

computer-vision benchmarks

arXiv AI

Jun 4

SymTRELLIS: Symmetry-Enforced Voxel Latents for 3D Generation

arXiv:2606. 04108v1 Announce Type: cross Abstract: Single-view 3D generative models have achieved impressive visual quality, yet they are not designed to satisfy structural or functional requirements, and in practice, often fall short.

By Guangda Ji, Qimin Chen, Qinchan Li, Mingrui Zhao, Kai Wang, Hao Zhang

robotics benchmarks

arXiv AI

Jun 4

Beyond Objective Equivalence: Constraint Injection for LLM-Based Optimization Modeling on Vehicle Routing Problems

arXiv:2606. 04816v1 Announce Type: new Abstract: Large language models (LLMs) increasingly translate natural-language optimization problems into executable solver code.

By Xizi Luo, Changhong He, Dongdong Geng, Chenggong Shi, Yu Mei

llms benchmarks

arXiv Machine Learning

Jun 4

RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression

arXiv:2511. 21035v2 Announce Type: replace Abstract: Holography offers significant potential for AR/VR applications.

By Shima Rafiei, Zahra Nabizadeh Shahr-Babak, Soroush Khoubyarian, Alexandre Cooper, Shadrokh Samavi, Shahram Shirani

efficiency benchmarks

arXiv Machine Learning

Jun 4

MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers

arXiv:2606. 04366v1 Announce Type: new Abstract: Conventional patchified Transformers operate on uniform spatial partitions, distributing computational effort evenly across the domain irrespective of local features.

By Yanshun Zhao, Xiaoyu Peng, Jiamin Jiang, Congcong Zhu, Jingrun Chen

llms nlp benchmarks safety

arXiv AI

Jun 4

Supportive Token Revealing for Fast Diffusion Language Model Decoding

arXiv:2606. 04236v1 Announce Type: cross Abstract: Discrete diffusion language models can generate text efficiently by updating multiple masked positions in parallel, but this parallelism introduces a quality-latency trade-off.

By Giries Abu Ayoub, Mario Barbara, Llu\'is Pastor-P\'erez, Tanja Bien, Aneesh Barthakur, Alaa Maalouf, Loay Mualem

llms diffusion benchmarks

arXiv Machine Learning

Jun 4

LMM-IR: Large-Scale Netlist-Aware Multimodal Framework for Static IR-Drop Prediction

arXiv:2511. 12581v2 Announce Type: replace Abstract: Static IR drop analysis is a fundamental and critical task in the field of chip design.

By Kai Ma, Zhen Wang, Hongquan He, Qi Xu, Tinghuan Chen, Hao Geng

llms diffusion multimodal benchmarks

arXiv Machine Learning

Jun 4

Post-Training Corrections for Improved Time-Series Forecasting

arXiv:2505. 15354v3 Announce Type: replace Abstract: Time-series forecasting is a critical task in various business domains, but it remains inherently challenging.

By Hamza Cherkaoui, Malik Tiomoko, Giuseppe Paolo, Zhang Yili, Yu Meng, Zhang Keli, Hafiz Tiomoko Ali

benchmarks

arXiv Machine Learning

Jun 4

Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learning

arXiv:2606. 04492v1 Announce Type: new Abstract: Cooperative Multi-Agent Reinforcement Learning (MARL) frequently suffers from severe reward sparsity and exploration bottlenecks.

By Zicheng Zhao, Yu Lan, Chengzhengxu Li, Zhaohan Zhang, Xiaoming Liu

agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 4

VentAgent: When LLMs Learn to Breathe -- Multi-Objective Arbitration for ARDS Ventilation

arXiv:2606. 04632v1 Announce Type: new Abstract: Mechanical ventilation for Acute Respiratory Distress Syndrome (ARDS) requires balancing competing physiological goals, including oxygenation, lung protection, and acid-base homeostasis.

By Teqi Hao, Yuxuan Fu, Xiaoyu Tan, Shaojie Shi, Bohao Lv, Yinghui Xu, Xihe Qiu

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 4

DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities

arXiv:2606. 04205v1 Announce Type: cross Abstract: The growing popularity and capacity of generative models have eroded the distinction between human and machine-generated content, motivating a growing body of work on detection across text, images, and audio.

By Sajad Ebrahimi, Nima Jamali, Bardia Shirsalimian, Kelly McConvey, Wentao Zhang, Jalehsadat Mahdavimoghaddam, Maksym Taranukhin, Maura Grossman, Vered Shwartz, Yuntian Deng, Ebrahim Bagheri

multimodal benchmarks

arXiv AI

Jun 4

Expectations vs. Realities: The Cost of MSE-Optimal Forecasting Under Conditional Uncertainty

arXiv:2606. 04342v1 Announce Type: cross Abstract: Multi-step time series forecasting (MSF) is commonly evaluated using point-wise error metrics such as mean squared error (MSE), implicitly treating the conditional mean as a sufficient target.

By Riku Green, Zahraa S. Abdallah, Telmo M Silva Filho

benchmarks

arXiv Machine Learning

Jun 4

Learning Randomized Reductions

arXiv:2412. 18134v5 Announce Type: replace Abstract: Randomized self-reductions (RSRs) express $f(x)$ using $f$ evaluated at random correlated points, enabling self-correcting programs, instance-hiding protocols, and applications in complexity theory and cryptography.

By Ferhat Erata, Orr Paradise, Thanos Typaldos, Timos Antonopoulos, ThanhVu Nguyen, Shafi Goldwasser, Ruzica Piskac

llms agents benchmarks

arXiv Machine Learning

Jun 4

Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning

arXiv:2603. 09803v2 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance.

By Tiehua Mei, Minxuan Lv, Leiyu Pan, Zhenpeng Su, Hongru Hou, Hengrui Chen, Ao Xu, Deqing Yang

llms reinforcement-learning benchmarks

Get the daily briefing