Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

22,912 stories · RSS feed

arXiv AI

Jun 9

Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering

arXiv:2602. 17911v3 Announce Type: replace-cross Abstract: Current biomedical question answering (QA) systems often assume that medical knowledge applies uniformly, yet real-world clinical reasoning is inherently conditional: nearly every decision depends on patient-specific factors such as comorbidities and contraindications.

By Jash Rajesh Parekh, Wonbin Kweon, Joey Chan, Rezarta Islamaj, Robert Leaman, Pengcheng Jiang, Chih-Hsuan Wei, Zhizheng Wang, Zhiyong Lu, Jiawei Han

rag nlp benchmarks

arXiv AI

Jun 9

Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards

arXiv:2605. 03862v4 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards has become a common way to improve explicit reasoning in large language models, but final-answer correctness alone does not reveal whether the reasoning trace is faithful, reliable, or useful to the model that consumes it.

By Tianyang Han, Hengyu Shi, Junjie Hu, Xu Yang, Zhiling Wang, Junhao Su

llms reinforcement-learning benchmarks

arXiv AI

Jun 9

VideoGPA: Distilling Geometry Priors for 3D-Consistent Video Generation

arXiv:2601. 23286v4 Announce Type: replace-cross Abstract: While recent video diffusion models (VDMs) produce visually impressive results, they fundamentally struggle to maintain 3D structural consistency, often resulting in object deformation or spatial drift.

By Hongyang Du, Junjie Ye, Xiaoyan Cong, Runhao Li, Jingcheng Ni, Aman Agarwal, Zeqi Zhou, Zekun Li, Randall Balestriero, Yue Wang

diffusion reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 9

Alcmean's: Unsupervised community detection using local Laplacian, automatic detection of the number of centers

arXiv:2606. 09100v1 Announce Type: cross Abstract: Community detection is a fundamental problem in the analysis of complex networks.

By Shahin Momenzadeh, Rojiar Pir Mohammadiani

rag benchmarks

arXiv AI

Jun 9

Chiaroscuro Attention: Spending Compute in the Dark

arXiv:2606. 08327v1 Announce Type: cross Abstract: Standard transformers apply self-attention uniformly at every layer and token, regardless of whether the input requires dynamic cross-token interaction.

By Prateek Kumar Sikdar

llms benchmarks

arXiv AI

Jun 9

LGMT: Logic-Grounded Metamorphic Testing for Evaluating the Reasoning Reliability of LLMs

arXiv:2605. 23965v2 Announce Type: replace Abstract: Large Language Models (LLMs) achieve strong performance on logical reasoning benchmarks, yet their reliability remains uncertain.

By Zenghui Zhou, Man Li, Xiaoke Fang, Xinyi Zhou, Weibin Lin, Zheng Zheng

llms benchmarks

arXiv AI

Jun 9

CoVEBench: Can Video Editing Models Handle Complex Instructions?

arXiv:2606. 08415v1 Announce Type: cross Abstract: While recent text-guided video editing models excel at elementary tasks (e.

By Jiangtao Wu, Jiaming Wang, Yiwen He, Yuanxing Zhang, Shihao Li, Dunyuan Liu, Xuedong Zhao, Jialu Chen, Zekun Moore Wang, Jiaheng Liu

benchmarks

arXiv AI

Jun 9

DYCP: Dynamic Context Pruning for Long-Form Dialogue with LLMs

arXiv:2601. 07994v5 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly operate over long-form dialogues with frequent topic shifts.

By Nayoung Choi, Jonathan Zhang, Jinho D. Choi

llms efficiency benchmarks

arXiv AI

Jun 9

See More, Think Deeper: Query-Expanded Visual Evidence and Answer-Clue Guided Reflection for Long Video Understanding

arXiv:2606. 09064v1 Announce Type: cross Abstract: Recent advances in Video Large Language Models (Video-LLMs) have enabled performance on long-video understanding tasks.

By Shuning Wang, Zhiheng Wu, YiNuo Lu, Naiming Liu, Chen Jia, Bowen Liu, Shuo Nie, Weijie Zhu, Yumeng Zhang

llms benchmarks

arXiv Machine Learning

Jun 9

Curvature-Guided LoRA: Matching Full Fine-Tuning in Function Space

arXiv:2603. 29824v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning methods such as LoRA enable efficient adaptation of large pretrained models, but often lag behind full fine-tuning in both convergence speed and final performance.

By Fr\'ed\'eric Zheng, Alexandre Prouti\`ere

fine-tuning benchmarks safety

arXiv AI

Jun 9

SafeECGMatch: Calibration-Aware Joint Frequency and Time Space Semi-Supervised Learning for Open-Set ECG Classification

arXiv:2606. 08037v1 Announce Type: cross Abstract: Electrocardiogram (ECG) classification models often suffer from severe label scarcity, making semi-supervised learning (SSL) an attractive strategy for reducing annotation costs.

By Hongkyu Koh, Ikbeom Jang

benchmarks

arXiv AI

Jun 9

FuseFSS: Efficient Secure LLM Inference with Function Secret Sharing

arXiv:2606. 09551v1 Announce Type: cross Abstract: Two-server secure inference allows a client to query a hosted large language model (LLM) without revealing prompts or embeddings.

By Yuhan Ma, Yong Li, Stefan Schmid

llms rag benchmarks

arXiv Machine Learning

Jun 9

SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling

arXiv:2606. 09304v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own trajectories with dense per-token supervision from a stronger teacher, and often outperforms off-policy distillation and standard reinforcement learning.

By Haoran Xu, Hongyu Wang, Yifei Gao, Jiaze Li, Xiaofeng Zhang, Xiaosong Yuan

reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jun 9

Deep reinforcement learning for process design: Review and perspective

arXiv:2308. 07822v2 Announce Type: replace Abstract: The transformation towards renewable energy and feedstock supply in the chemical industry requires new conceptual process design approaches.

By Qinghe Gao, Artur M. Schweidtmann

agents reinforcement-learning benchmarks

arXiv AI

Jun 9

Structured Neuron Pruning in Deep Neural Networks Using Multi-Armed Bandits

arXiv:2606. 07615v1 Announce Type: cross Abstract: Deep neural networks often contain redundant hidden units.

By Salem Ameen, Sunil Vadera

reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 9

Improving the sharpness in neural network-based parametric post-processing of ensemble forecasts

arXiv:2606. 08587v1 Announce Type: cross Abstract: Statistical post-processing has proven to be an effective tool in improving ensemble forecast of different weather variables.

By \'Agnes Baran, M\'at\'e Mihalina

benchmarks

arXiv AI

Jun 9

Intelligent Character Recognition of Handwritten Forms with Deep Neural Networks

arXiv:2606. 08858v1 Announce Type: cross Abstract: The automatic processing of handwritten forms remains a challenging task, wherein detection and subsequent classification of handwritten characters are essential steps.

By Hartwig Grabowski

benchmarks

arXiv Machine Learning

Jun 9

KITE: A Tri-Modal Transformer Integrating Text, Images, and Knowledge Graphs for Fake News Detection

arXiv:2606. 07651v1 Announce Type: new Abstract: Traditional fake news detection methods are falling behind as multimodal misinformation grows more advanced, seamlessly blending deceptive text, manipulated visuals, and factually incorrect claims.

By Kevin Patel, Shashi Bhushan Jha

llms multimodal benchmarks safety

arXiv AI

Jun 9

Deterministic Integrity Gates for LLM-Assisted Clinical Manuscript Preparation: An Auditable Biomedical Informatics Architecture

arXiv:2606. 09500v1 Announce Type: new Abstract: Objective.

By Yoojin Nam, Jinhoon Jeong, Namkug Kim

llms benchmarks

arXiv Machine Learning

Jun 9

Decomposable Neuro Symbolic Regression

arXiv:2511. 04124v3 Announce Type: replace Abstract: Symbolic regression (SR) models complex systems by discovering mathematical expressions that capture underlying relationships in observed data.

By Giorgio Morales, John W. Sheppard

llms benchmarks

Get the daily briefing