Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv AI

Jun 4

Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text

arXiv:2605. 29076v2 Announce Type: replace-cross Abstract: LLMs have advanced text classification, yet existing paradigms face a trade-off: supervised (label only) fine-tuning is scalable but offers limited reasoning on complex text and lacks broader model transparency, while discrete prompt optimization offers human-readable instructions but struggles with performance and scalability.

By Tianyang Zhou, Wenbo Chen, Pierre Jinghong Liang, Leman Akoglu

llms nlp reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv Machine Learning

Jun 4

CaloTrilogy: Toward a Breakthrough in One-Step, End-to-End, Physics-Guided Shower Generation for Modern Calorimeters

arXiv:2606. 04165v1 Announce Type: cross Abstract: High-precision calorimeter simulation at current and future colliders imposes rapidly growing computational demands, motivating the development of machine-learning surrogates for traditional Monte Carlo tools such as Geant4.

By Cheng Jiang, Sitian Qian, Kevin Pedro, Oz Amram, Huilin Qu, Maggie Voetberg

diffusion benchmarks

arXiv AI

Jun 4

Rollout-Level Advantage-Prioritized Experience Replay for GRPO

arXiv:2606. 04560v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards with GRPO is a standard approach for post-training reasoning LLMs.

By Gyeongtae Yoo, Sanghyeok Park, Soohyuk Jang, Ik-hwan Kim, Sungroh Yoon

llms reinforcement-learning benchmarks

arXiv AI

Jun 4

Adaptive Calibration for Fair and Performant Facial Recognition

arXiv:2606. 04469v1 Announce Type: cross Abstract: We introduce Adaptive Calibration (AC), a novel calibration strategy for facial recognition that maps cosine similarity between normalized embeddings to well-calibrated probabilities.

By Ryan Brown, Chris Russell

rag benchmarks safety

arXiv AI

Jun 4

Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models

arXiv:2606. 04326v1 Announce Type: cross Abstract: Concept bottleneck models predict outcomes from high-level concepts detected in inputs.

By Julian Skirzynski, Harry Cheon, Shreyas Kadekodi, Meredith Stewart, Berk Ustun

benchmarks safety

arXiv Machine Learning

Jun 4

Geometry-Preserving Encoder/Decoder in Latent Generative Models

arXiv:2501. 09876v3 Announce Type: replace-cross Abstract: Generative modeling aims to generate new data samples that resemble a given dataset.

By Wonjun Lee, Riley C. W. O'Neill, Dongmian Zou, Jeff Calder, Gilad Lerman

diffusion benchmarks

arXiv AI

Jun 4

Selective Coupling of Decoupled Informative Regions: Masked Attention Alignment for Data-Free Quantization of Vision Transformers

arXiv:2606. 04373v1 Announce Type: cross Abstract: Data-Free Quantization (DFQ) addresses data security concerns by synthesizing samples, without accessing real data.

By Biao Qian, Yang Wang, Yong Wu, Jungong Han

llms efficiency benchmarks safety

arXiv Machine Learning

Jun 4

FoeGlass: Simple In-Context Learning Is Enough for Red Teaming Audio Deepfake Detectors

arXiv:2606. 05101v1 Announce Type: cross Abstract: Audio deepfake detection (ADD) models are critical for countering the malicious use of text-to-speech (TTS) models.

By Sepehr Dehdashtian, Jacob H Seidman, Vishnu N Boddeti, Gaurav Bharaj

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jun 4

Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models

arXiv:2602. 19101v2 Announce Type: replace-cross Abstract: Value alignment of Large Language Models (LLMs) requires us to empirically measure these models' actual, acquired representation of value.

By Seong Hah Cho, Junyi Li, Anna Leshinskaya

llms rag benchmarks safety

arXiv Machine Learning

Jun 4

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

arXiv:2606. 04613v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) struggle when applied to medical image-text data, yet the tools available to diagnose this failure remain limited.

By Alessandro Gambetti, Qiwei Han, Cl\'audia Soares, Hong Shen

llms multimodal benchmarks safety

arXiv AI

Jun 4

SFMambaNet: Spectral-Frequency Enhanced Selective State Space Model for Correspondence Pruning

arXiv:2606. 04493v1 Announce Type: cross Abstract: Correspondence pruning aims to identify inliers from an initial set of correspondences.

By Zhihua Wang, Yanping Li, Yizhang Liu

efficiency benchmarks

arXiv AI

Jun 4

Learning to Evaluate: Cost-Effective Model Evaluation on Unlabeled Data with Meta-Learning

arXiv:2605. 23595v2 Announce Type: replace-cross Abstract: The rapid advancement of machine learning has led to an unprecedented expansion of model ecosystems, making it increasingly difficult to assess the reliability of newly released models on unseen and unlabeled data.

By Trinh Pham, Viet Huynh, Hongzhi Yin, Quoc Viet Hung Nguyen, Thanh Tam Nguyen

fine-tuning benchmarks

arXiv AI

Jun 4

Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

arXiv:2605. 24602v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood.

By Quanjiang Li, Zhiming Liu, Wei Luo, Tingjin Luo, Chenping Hou

llms multimodal benchmarks safety

arXiv AI

Jun 4

Metric-Aware Hybrid Forecasting for the CTF4Science Lorenz Challenge

arXiv:2606. 04191v1 Announce Type: cross Abstract: We describe our approach to the CTF4Science Lorenz challenge, a benchmark that mixes short-horizon forecasting, long-time distribution matching, and trajectory reconstruction across nine task pairs.

By Cen Lu

benchmarks

arXiv AI

Jun 4

Temporal Order Matters for Agentic Memory: Segment Trees for Long-Horizon Agents

arXiv:2606. 04555v1 Announce Type: cross Abstract: Long-horizon conversational agents need to interact with users through evolving events, tasks, and goals.

By Yifan Simon Liu, Liam Gallagher, Faeze Moradi Kalarde, Jiazhou Liang, Armin Toroghi, Scott Sanner

llms agents benchmarks

arXiv AI

Jun 4

Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning

arXiv:2505. 17315v2 Announce Type: replace Abstract: Recent language models exhibit strong reasoning capabilities, yet the influence of long-context capacity on reasoning remains underexplored.

By Wang Yang, Zirui Liu, Hongye Jin, Qingyu Yin, Vipin Chaudhary, Xiaotian Han

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 4

Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling

arXiv:2606. 05021v1 Announce Type: new Abstract: We investigate multi-agent deep reinforcement learning and propose two enhancements to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm.

By Marc Walden, Jason Liu, Shaashwath Sivakumar, Ryan Liu, Hamza Khan

agents reinforcement-learning benchmarks

arXiv AI

Jun 4

'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions

arXiv:2606. 04906v1 Announce Type: cross Abstract: Although it is generally agreed that AI-generated text poses a broad societal risk, there is no common understanding in the AI-generated text detection literature on what constitutes harmful use.

By Nils Dycke, Marina Sakharova, Nico Daheim, Iryna Gurevych

benchmarks

arXiv Machine Learning

Jun 4

Graph Cascades: Contagion-Based Mesoscopic Rewiring for Structure-Aware Graph Machine Learning

arXiv:2606. 05046v1 Announce Type: new Abstract: We introduce Graph Cascades, a mesoscopic rewiring strategy for Graph Neural Networks (GNNs) and Graph Transformers (GTs) that captures intermediate-scale graph structure beyond purely local edges or fully global attention.

By Meher Chaitanya, My Le, Luana Ruiz

llms diffusion benchmarks

arXiv AI

Jun 4

Parthenon Law: A Self-Evolving Legal-Agent Framework

arXiv:2606. 04602v1 Announce Type: new Abstract: As agents grow more capable, legal-domain LLM agents promise to turn document-heavy matters into reviewable work products -- yet reliable deployment faces three obstacles: no large-scale evidence on how today's strongest model-and-harness combinations behave on end-to-end legal matters; no agent architecture adapted to the legal vertical, only general-purpose harnesses; and, in a setting that keeps shifting with new facts, authorities, and deadlines, no mechanism for systems to learn from their own outcomes.

By Hejia Geng, Leo Liu

llms agents benchmarks

Get the daily briefing