Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,633 stories · RSS feed

arXiv AI

Jul 21

LookME: Lookup-Based Multimodal Embeddings for Layer Injection in Vision-Language Models

arXiv:2607. 16305v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have achieved strong progress in multimodal understanding.

By Zeyu Xu, Xingzhong Hou, Pengkai Guo, Siling Lin, Xiao Xu, Menghua Zhai, Haoyu Chen, Yunke Zhang, Fei Huang

llms rag multimodal benchmarks

arXiv AI

Jul 21

Toward Anthropomorphic Dialogue: A Closed-Loop Framework for Human-Like Chat Generation, Evaluation, and Preference Alignment

arXiv:2607. 17191v1 Announce Type: new Abstract: Human-like private chat requires more than fluent response generation: a system must preserve persona, relationship, memory, bounded knowledge, medium-specific timing, and a coherent multi-turn arc.

By Wentao Liu, Siyu Song, Xi Chen, Youjia Li, Xiaokun Wang, Min Ji, Ji Wang

reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 21

PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark

arXiv:2607. 16268v1 Announce Type: new Abstract: Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence.

By Ali Sultonov

llms diffusion nlp benchmarks

arXiv Machine Learning

Jul 21

Leakage-Robust Evaluation and Data-Scale Sensitivity of Attention-Enhanced Multi-Task Learning for Joint Fault Diagnosis and Remaining Useful Life Estimation

arXiv:2607. 16493v1 Announce Type: new Abstract: Multi-task deep learning models that jointly perform fault classification and remaining useful life (RUL) regression are increasingly used in predictive maintenance, yet reported performance can be strongly affected by how sliding-window sequences are split into training and test sets.

By Md Mahamudur Rahaman Shamim, Md. Nuruzzaman, Zannatul Ferdus, Md Rajib Ahmed, Abieer Nwshad Anward, Mohammad Tooneer, Johir Uddin Khan, Khalid Hossen

benchmarks

arXiv Machine Learning

Jul 21

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

arXiv:2607. 16721v1 Announce Type: new Abstract: The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token.

By Anik Jha

llms agents fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 21

A Novel Hybrid Quantum Reservoir Computing (nHQRC) for Phase Transition Detection in Non-Equilibrium Dynamical Systems

arXiv:2607. 16281v1 Announce Type: cross Abstract: The analysis of highly non-linear stochastic data within non-equilibrium dynamical systems requires computational frameworks capable of detecting latent phase transitions before systemic structural breakdowns occur.

By Manoj B. Bhatkar, Prashant M. Yawalkar

diffusion benchmarks

arXiv Machine Learning

Jul 21

Diagnosing Correctness Probes under Self-Judgement Confounding

arXiv:2607. 16799v1 Announce Type: cross Abstract: Hidden-state readouts can predict whether language-model outputs are correct, but objective correctness (OC) usually agrees with the model's own self-judgement (SJ), leaving the decoded signal semantically ambiguous.

By Yi-Long Lu

benchmarks

arXiv Machine Learning

Jul 21

Depth-Regularized JEPA World Models Learn More Transferable Representations from Real Outdoor Robot Data

arXiv:2607. 16314v1 Announce Type: cross Abstract: World models, especially based on JEPA architectures, have been shown to learn robust dynamics of various environments.

By Usman M. Khan

robotics benchmarks

arXiv AI

Jul 21

Constraint-Anchored Reasoning Traces

arXiv:2607. 16727v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) suffer from error snowballing: a single incorrect inference early in a chainof-thought (CoT) trace corrupts all downstream reasoning.

By Zehua Cheng, Wei Dai, Jiahao Sun

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 21

Quantifying Diversity of Thought: A Predictive Law of Weighted LLM Ensemble Lift

arXiv:2607. 17384v1 Announce Type: new Abstract: This paper provides an experimentally verified formal law for calculating the uplift that diversity of thought provides in Large Language Model (LLM) ensembles.

By Junade Ali

llms agents benchmarks

arXiv AI

Jul 21

Lomekwi: Resource-Bounded Tool Discovery in LLM Agents

arXiv:2607. 16961v1 Announce Type: new Abstract: Existing tool-use benchmarks report a single success rate for complex, multistep tasks.

By Roshan Klein-Seetharaman, Daniel Wang, Andrew Xu

llms agents benchmarks

arXiv AI

Jul 21

Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models

arXiv:2607. 17779v1 Announce Type: new Abstract: Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly in generating Not-Safe-For-Work (NSFW) images.

By Dongdong Yang, Deyue Zhang, Zhao Liu, Zonghao Ying, Wenzhuo Xu, Jiankai Jin, Xiangzheng Zhang, Quanchen Zou

llms diffusion multimodal benchmarks safety

arXiv Machine Learning

Jul 21

ECG-LLM: Foundation Model for ECG-Based Cardiac Reasoning

arXiv:2607. 16323v1 Announce Type: cross Abstract: Electrocardiography (ECG) is an inexpensive, standard-of-care test for cardiac symptoms, but front-line triage often lacks immediate access to definitive imaging such as echocardiography (ECHO) or cardiac magnetic resonance (CMR).

By Alexander Selivanov, Friederike Jungmann, Jan Kehrer, Karl-Ludwig Laugwitz, Eimo Martens, Daniel Rueckert

llms multimodal benchmarks

arXiv AI

Jul 21

ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video

arXiv:2607. 17790v1 Announce Type: cross Abstract: Egocentric devices, such as wearable front-facing cameras, provide a unique perspective for capturing the continuous interaction between a human viewer and the surrounding environment.

By Xiaozhong Lyu, Gen Li, Zhiyin Qian, Xucong Zhang, Marc Pollefeys, Siyu Tang

llms computer-vision efficiency multimodal benchmarks

arXiv Machine Learning

Jul 21

A Framework for Early Sepsis Prediction via Self-Supervised (JEPA) and Federated Representation Learning

arXiv:2607. 16681v1 Announce Type: new Abstract: Early sepsis prediction from electronic health records is challenged by irregular sampling, high missingness, and class imbalance.

By Umair bin Mansoor, Munaf Rashid, Roomi Naqvi

rag fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 21

Spectral-Morphological Attention U-Net: An Efficient Network for Active Wildfire Detection

arXiv:2607. 16472v1 Announce Type: cross Abstract: Over the past decades, the frequency of global wildfires has been increasing steadily.

By Yugong Zeng, Jonathan Wu

computer-vision benchmarks

arXiv Machine Learning

Jul 21

Backpropagation-Free Trunk Training via the Split Forward Gradients

arXiv:2607. 16612v1 Announce Type: cross Abstract: Backpropagation makes training deep networks memory intensive because it must store intermediate activations.

By Tian Qin, Wei-Min Huang

llms benchmarks

arXiv Machine Learning

Jul 21

Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection

arXiv:2607. 17382v1 Announce Type: cross Abstract: Existing research shows that AI-generated text detection classifiers achieve strong in-distribution (ID) performance but do not maintain the same performance on out-of-distribution (OOD) texts, suggesting overfitting to dataset-specific features.

By Shantanu Thorat

fine-tuning benchmarks

arXiv AI

Jul 21

RobustMAD: Evaluating Real-World Robustness of Multimodal Small Language Models for Deployable Anomaly Detection Assistants

arXiv:2607. 16243v1 Announce Type: cross Abstract: Multimodal industrial anomaly inspection assistants are a critical component of next-generation smart factories, enabling interactive vision-language-based querying.

By Anushiya Arunan, Xin Li, Yan Qin, U-Xuan Tan, Nhu Khue Vuong, Xiaoli Li, Chau Yuen

llms multimodal benchmarks safety

arXiv AI

Jul 21

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

arXiv:2607. 17761v1 Announce Type: cross Abstract: Recently, speech deepfake detection (SDD) has achieved significant progress.

By Jun Xue, Zhuolin Yi, Yanzhen Ren, Yihuan Huang, Jiayu Xiong, Yi Chai, Guanxiang Feng, Jiajun Liu, Tong Zhang

benchmarks safety

Get the daily briefing