Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,065 stories · RSS feed

arXiv AI

Jul 22

MILP-Evo: Closed-Loop Fully Automatic Design of MILP Solvers

arXiv:2607. 18252v1 Announce Type: new Abstract: Machine learning methods have shown that data-driven policies can accelerate mixed-integer linear programming (MILP) solvers, but many such approaches remain difficult to inspect, adapt, and deploy because the learned policy is represented as an external predictor or other opaque model.

By Jinbiao Nie, Kewei Feng, Xiaoyuan Zhang, Shan Yin, Zizhuo Wang, Bin Dong

llms benchmarks

arXiv AI

Jul 22

Dr. Zero: Self-Evolving Search Agents without Training Data

arXiv:2601. 07055v2 Announce Type: replace Abstract: As high-quality data becomes increasingly difficult to obtain, self-evolution without curated training data has emerged as a promising paradigm.

By Zhenrui Yue, Kartikeya Upasani, Xianjun Yang, Suyu Ge, Shaoliang Nie, Yuning Mao, Zhe Liu, Dong Wang

llms agents nlp benchmarks

arXiv AI

Jul 22

Towards Principled Continual Anomaly Detection: A Systematic Framework and Benchmark Scenarios

arXiv:2607. 18289v1 Announce Type: cross Abstract: Continual anomaly detection (CAD) studies how models can adapt to evolving data distributions while retaining performance on previously observed regimes.

By Kamil Faber, Mateusz Smendowski, Roberto Corizzo

benchmarks

arXiv AI

Jul 22

Agents in the Wild: Where Research Meets Deployment

arXiv:2607. 19336v1 Announce Type: new Abstract: Agentic systems large language model (LLM) based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents are rapidly transitioning from research prototypes to production scale deployments across domains such as software engineering, scientific discovery, and finance.

By Grace Hui Yang, Pranav N. Venkit, Hooman Sedghamiz, Enrico Santus, Victor Dibia, Ioana Baldini

llms agents benchmarks safety

arXiv AI

Jul 22

Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interactio

arXiv:2607. 18985v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in language understanding, reasoning, and world knowledge.

By Jialian Li, Junhong Liu, Yuchen Cao, Weiran Guo, Jiaming Song, Xutao Wang, Yi Zhao, Jiangpin Liu, Jie Chen

llms agents reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jul 22

Evaluating medical AI under missing information: same-provider judges and human raters change apparent safety

arXiv:2607. 18828v1 Announce Type: new Abstract: Readiness stress-testing of medical AI has focused on closed-ended and multimodal benchmarks.

By Koyar Afrasyab

llms multimodal benchmarks safety

arXiv AI

Jul 22

Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models

arXiv:2607. 18280v1 Announce Type: cross Abstract: Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation, yet aggressive sparsification can trigger rapid performance degradation beyond an essential sparsity boundary.

By Chao Han, Haozhe Hu, Xiaoyu Shen

llms nlp efficiency benchmarks

arXiv AI

Jul 22

Relay-Bench: Evaluating LLMs on Multi-Domain Reasoning Chains

arXiv:2607. 18438v1 Announce Type: cross Abstract: Introducing Relay-Bench, an unsaturated, holistic, text-only benchmark that measures LLMs' ability to complete an assortment of tasks from distinct domains in a single prompt.

By Liam Swayne

llms multimodal benchmarks

arXiv AI

Jul 22

MUX: Continuous Reasoning via Multiplexed Tokens

arXiv:2607. 18264v1 Announce Type: new Abstract: Language models solve complex problems by articulating intermediate reasoning steps in natural language.

By Ayhan Suleymanzade, Halil Alperen Gozeten, Michael Bronstein, \.Ismail \.Ilkan Ceylan, Jinwoo Kim

llms diffusion efficiency benchmarks

arXiv AI

Jul 22

BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment

arXiv:2607. 05462v2 Announce Type: replace-cross Abstract: As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse.

By Edwin H. Wintermute, Harmon Bhasin, Christina M. Agapakis, Dianzhuo Wang, Evan Seeyave, Arjun Banerjee, Daniel Fulop, Matthew C. Watson, Adam J. Meyer, Sandrine Boissel, Jens H. Kuhn, Rishi Jain, Noah D. Taylor, Helena Shomar, Patrick M. Boyle, Kenny Workman

agents benchmarks safety

arXiv Machine Learning

Jul 22

Data Reliability Scoring

arXiv:2510. 17085v2 Announce Type: replace Abstract: How can we assess the reliability of a dataset without access to ground truth?

By Yiling Chen, Shi Feng, Paul Kattuman, Fang-Yi Yu

rag benchmarks data-engineering

arXiv AI

Jul 22

Edge-Efficient Transformer for End-to-End RF Spectrum Monitoring

arXiv:2607. 18285v1 Announce Type: cross Abstract: We present E-SpecFormer (Edge Spectrum monitoring Transformer) for end-to-end automatic modulation and covert channel (CC) recognition.

By Zhifan Song, Haralampos-G. Stratigopoulos, Hassan Aboushady

llms benchmarks

arXiv AI

Jul 22

FSDBN: Foreground-Aware EEG--Visual Alignment via Dynamic Brain Networks

arXiv:2607. 18344v1 Announce Type: cross Abstract: EEG-based visual decoding provides a non-invasive pathway for interpreting visual semantics.

By Yiheng Liu, Chuhang Zheng, Peiliang Gong, Jingtao Liu, Daoqiang Zhang, Qi Zhu

benchmarks safety

arXiv AI

Jul 22

Frontier LLM-based agents can overcome the ontology curation bottleneck for natural phenotypes

arXiv:2605. 28965v2 Announce Type: replace Abstract: Linking free-text phenotype descriptions to ontology terms, typically referred to as phenotype annotation, is essential for the cross-study integration of comparative morphological data.

By James P. Balhoff, Hilmar Lapp

llms agents nlp benchmarks

arXiv AI

Jul 22

Fishing Out Free Riders: Shapley-Based Reward Attribution for Parallel Reasoning via Reinforcement Learning

arXiv:2607. 18979v1 Announce Type: new Abstract: Large Language Models (LLMs) excel at multi-step reasoning, yet current parallel reasoning approaches often fail to distinguish the contributions of individual reasoning paths.

By Wentao Zhang, Haoyu Zhang, Xinke Jiang, Yuxuan Cheng, Yuhan Pan, Miao Li, Zhipeng Qiao, Tao Feng, Zhen Tao, Dengji Zhao

llms reinforcement-learning benchmarks

arXiv AI

Jul 22

Saving the legacy of Hero Ibash: Evaluating Four Language Models for Aminoacian

arXiv:2402. 18121v2 Announce Type: replace-cross Abstract: This study assesses four cutting-edge language models in the underexplored Aminoacian language.

By Yunze Xiao, Yiyang Pan

llms nlp benchmarks

arXiv AI

Jul 22

Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding

arXiv:2607. 18909v1 Announce Type: new Abstract: This paper explores Relative Positional Encoding (RPE) as an additive bias in Transformer architectures to solve the Team Orienteering Problem.

By Leyre Enc\'io, Daniel Fuertes, Carlos R. del-Blanco, Fernando Jaureguizar

llms rag benchmarks safety

arXiv AI

Jul 22

ChemHyperMag: Physics-informed magnetic hypergraph learning improves molecular ADMET prediction

arXiv:2607. 18332v1 Announce Type: cross Abstract: Accurate prediction of ADMET (Absorption, Distribution, Metabolism, Excretion, and Toxicity) is important for drug discovery.

By Hexiao Ding, Hongzhao Chen, Jing Lan, Yufeng Jiang, Zihong Luo, Zehua Xiong, Tianlong Ruan, Yunlin Mao, Nga Chun Ng, Gwing Kei Yip, Gerald W. Y. Cheng, Kate Inyoung Oh, Jing Cai, Liang-Ting Lin, Jung Sun Yoo

benchmarks safety

arXiv Machine Learning

Jul 22

In-Context Time Series Classification with Random Convolutional Features

arXiv:2607. 19234v1 Announce Type: new Abstract: Time series classification is central to domains like medical signal analysis, industrial monitoring, and sensor-based activity recognition, where class information manifests as localized shapes, specific frequencies, temporal shifts, or complex cross-channel interactions.

By Joscha C\"uppers, Jilles Vreeken

benchmarks

arXiv AI

Jul 22

HALLMARK: Diagnosing Three Failure Modes in LLM Citation Verifiers

arXiv:2607. 18360v1 Announce Type: cross Abstract: Large language models (LLMs) now routinely draft literature reviews and assist with academic writing, which means a higher risk of fabricated references: GPTZero found 53 papers with hallucinated citations among NeurIPS 2025's accepted set.

By Patrik Reizinger, Wieland Brendel

llms agents benchmarks safety

Get the daily briefing