Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,477 stories · RSS feed

arXiv Machine Learning

Jun 5

Next-Generation Parallel Decoder for LPDR: Architectural Optimization and Class-Balanced GAN-Augmentation

arXiv:2606. 05785v1 Announce Type: cross Abstract: Real-Time License Plate Detection and Recognition (LPDR) forms the backbone of modern smart cities.

By Shawaiz Obaid, Nida Chandio, Neha Jamil, Muhammad Khuram Shahzad

diffusion benchmarks

arXiv Machine Learning

Jun 5

Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries

arXiv:2606. 05970v1 Announce Type: cross Abstract: Large language models are increasingly used for structured extraction from clinical free-text notes, but the sensitivity of their output to upstream configuration choices is less understood than their accuracy on fixed benchmarks.

By Martin Murin

llms benchmarks

arXiv Machine Learning

Jun 5

Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection

arXiv:2606. 06481v1 Announce Type: cross Abstract: As AI writing assistants become increasingly integrated into real-world drafting and revision workflows, many documents are no longer purely human-written or AI-generated, but instead result from progressive human-AI co-editing.

By Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Tianjun Yao, Xinyi Shang, Yi Tang, Jiacheng Cui, Ahmed Elhagry, Salwa K. Al Khatib, Hao Li, Salman Khan, Zhiqiang Shen

benchmarks

arXiv Machine Learning

Jun 5

IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures

arXiv:2604. 07709v4 Announce Type: replace-cross Abstract: A heavily safety-trained model will hand a physician the full, patient-followable benzodiazepine taper and refuse it to the patient who needs it, over identical clinical facts; the knowledge is present either way.

By David Gringras

llms benchmarks safety

arXiv Machine Learning

Jun 5

Quantifying Sensitivity for Tree Ensembles: A symbolic and compositional approach

arXiv:2605. 13830v2 Announce Type: replace-cross Abstract: Decision tree ensembles (DTE) are a popular model for a wide range of AI classification tasks, used in multiple safety critical domains, and hence verifying properties on these models has been an active topic of study over the last decade.

By Ajinkya Naik, Chaitanya Garg, S. Akshay, Ashutosh Gupta, Kuldeep S. Meel

benchmarks safety

arXiv Machine Learning

Jun 5

Binary Gaussian Copula Synthesis: an LLM-powered data augmentation framework for early dialysis prediction in chronic kidney disease

arXiv:2403. 00965v2 Announce Type: replace-cross Abstract: Only a small fraction of patients with chronic kidney disease (CKD) progress to dialysis, creating severe class imbalance that limits the performance of machine learning models for early dialysis prediction.

By Hamed Khosravi, Milad Khanchi, Mobina Noori, Srinjoy Das, Abdullah Al-Mamun, Imtiaz Ahmed

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 5

Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution

arXiv:2505. 11766v4 Announce Type: replace Abstract: Neural Operators (NOs) are powerful architectures for learning mappings between function spaces.

By Haoze Song, Zhihao Li, Xiaobo Zhang, Zecheng Gan, Zhilu Lai, Wei Wang

rag benchmarks

arXiv Machine Learning

Jun 5

A Survey on Diffusion Language Models

arXiv:2508. 10875v3 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) are rapidly emerging as a powerful and promising alternative to the dominant autoregressive (AR) paradigm.

By Tianyi Li, Mingda Chen, Bowei Guo, Zhiqiang Shen

llms diffusion nlp efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

arXiv:2510. 05544v2 Announce Type: replace-cross Abstract: Large language models (LLM) and vision-language models (VLM) have achieved state-of-the-art performance, but they impose significant memory and computing challenges in deployment.

By Ryan Solgi, Parsa Madinei, Jiayi Tian, Rupak Swaminathan, Jing Liu, Nathan Susanj, Zheng Zhang

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jun 5

Less is MoE: Trimming Experts in Domain-Specialist Language Models

arXiv:2606. 05538v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models achieve strong performance through conditional computation, but their large parameter footprint poses deployment challenges.

By Haoze He, Xinkai Zou, Xuan Jiang, Xingyuan Ding, Ao Qu, Juncheng Billy Li, Heather Miller

llms efficiency benchmarks

arXiv Machine Learning

Jun 5

Multi-Fidelity Learning with Shallow Recurrent Decoders for Reactor Physics

arXiv:2606. 05202v1 Announce Type: cross Abstract: In reactor physics, neutronics can be treated with different fidelity levels, according to the needs of the user.

By Stefano Riva, Carolina Introini, J. Nathan Kutz, Antonio Cammi

diffusion benchmarks

arXiv Machine Learning

Jun 5

DiffSlack: Learning under Nonlinear Inequality Constraints via Learnable Slack Variables

arXiv:2606. 05247v1 Announce Type: new Abstract: Enforcing nonlinear inequality constraints in neural networks remains challenging, especially when the output is subject to many coupled constraints.

By Ziqian Wang, Chenxi Fang, Zhen Zhang

rag benchmarks

arXiv Machine Learning

Jun 5

TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learning

arXiv:2606. 05878v1 Announce Type: new Abstract: Foundation models mark a profound paradigm shift in time series modeling, with task-specific models being superseded by general-purpose zero-shot models.

By Etienne Le Naour, Tahar Nabil, Adrien Petralia

llms benchmarks

arXiv Machine Learning

Jun 5

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

arXiv:2604. 01489v2 Announce Type: replace Abstract: High-performance GPU kernels are critical to modern machine learning systems, yet developing them remains a manual, expert-driven process.

By Tara Saba, Zhiyang Chen, Jikai Jason Li, Anne Ouyang, Xujie Si, Fan Long

llms agents benchmarks

arXiv Machine Learning

Jun 5

A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth

arXiv:2601. 21817v2 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) on open-ended tasks without ground-truth labels is increasingly done via the LLM-as-a-judge paradigm.

By Mingyuan Xu, Xinzi Tan, Jiawei Wu, Doudou Zhou

llms benchmarks

arXiv Machine Learning

Jun 5

Cluster-Aware Causal Mixer for Online Anomaly Detection in Multivariate Time Series

arXiv:2506. 00188v2 Announce Type: replace Abstract: Early and accurate detection of anomalies in time-series data is critical due to the substantial risks associated with false or missed detections.

By Md Mahmuddun Nabi Murad, Yasin Yilmaz

rag benchmarks

arXiv Machine Learning

Jun 5

Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation

arXiv:2606. 05988v1 Announce Type: new Abstract: Reasoning models produce long chain-of-thought traces that are costly to distill and encourage verbose student outputs.

By Maxime Griot, Paul Steven Scotti, Tanishq Mathew Abraham

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 5

CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction

arXiv:2606. 05799v1 Announce Type: new Abstract: Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's {\em behavioral robustness} to irrelevant or misleading information.

By Mohammad Anas Jawad, Cornelia Caragea

llms benchmarks

arXiv Machine Learning

Jun 5

Q-GNN: Query-Conditioned Graph Neural Networks with Type Awareness for Knowledge Graph Completion

arXiv:2606. 05639v1 Announce Type: new Abstract: Knowledge Graph Completion (KGC) aims at predicting missing triplets from incomplete knowledge graphs, which is crucial for downstream applications.

By Dongxiao He, Ruqiong Zhang, Zhizhi Yu, Ling Ding, Di Jin, Guangquan Xu, Zhiyong Feng

llms benchmarks

arXiv Machine Learning

Jun 5

LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges

arXiv:2605. 10807v4 Announce Type: replace-cross Abstract: The integration of Large Language Models (LLMs) into Electronic Design Automation (EDA) and hardware security is rapidly reshaping the semiconductor industry.

By Johann Knechtel, Ozgur Sinanoglu, Ramesh Karri

llms agents benchmarks safety

Get the daily briefing