Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,477 stories · RSS feed

arXiv AI

Jun 30

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

arXiv:2603. 29139v2 Announce Type: replace Abstract: Recent advances in large language models (LLMs) have enabled agentic systems to translate natural-language intent into executable scientific visualization (SciVis) tasks.

By Kuangshi Ai, Haichao Miao, Kaiyuan Tang, Nathaniel Gorski, Jianxin Sun, Guoxi Liu, Helgi I. Ingolfsson, David Lenz, Hanqi Guo, Hongfeng Yu, Teja Leburu, Michael Molash, Bei Wang, Tom Peterka, Chaoli Wang, Shusen Liu

llms agents multimodal benchmarks

arXiv AI

Jun 30

Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories

arXiv:2606. 28589v1 Announce Type: new Abstract: Current approaches to enhance Large Language Model (LLM) reasoning, such as Chain-of-Thought and "Wait" prompts, primarily encourage models to think more, yet often fail to guide them toward Truth.

By Tianlong Wang, Yuhang Wang, Weibin Liao, Xin Gao, Xinyu Ma, Yang Lin, Yasha Wang, Liantao Ma

llms benchmarks

arXiv Machine Learning

Jun 30

Shoot from the HIP: Hessian Interatomic Potentials without derivatives

arXiv:2509. 21624v3 Announce Type: replace Abstract: Fundamental tasks in computational chemistry, from transition state search to vibrational analysis, rely on molecular Hessians, which are the second derivatives of the potential energy.

By Andreas Burger, Luca Thiede, Nikolaj R{\o}nne, Varinia Bernales, Nandita Vijaykumar, Tejs Vegge, Arghya Bhowmik, Alan Aspuru-Guzik

benchmarks

arXiv AI

Jun 30

Physically-Constrained Harmonic Separation for Robust Heart and Respiratory Rate Estimation from Wrist Photoplethysmography

arXiv:2606. 30156v1 Announce Type: cross Abstract: Wrist-worn photoplethysmography (PPG) enables continuous monitoring of cardiopulmonary physiology, but reliable heart rate (HR) and respiratory rate (RR) estimation in free-living conditions remains challenging due to non-stationary motion artifacts that spectrally overlap with physiological dynamics.

By Nouhaila Fraihi, Ouassim Karrakchou, Mounir Ghogho

benchmarks safety

arXiv AI

Jun 30

Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

arXiv:2606. 29014v1 Announce Type: new Abstract: Recent advancements in generative artificial intelligence (AI) and large language models (LLMs) have shown significant promise in automating complex reasoning, summarization, and question-answering tasks.

By Dianwei Chen (Terry), Yuan-Zheng Lei (Terry), Zifan Zhang (Terry), Yuchen Liu (Terry), Xianfeng (Terry), Yang

llms agents nlp fine-tuning benchmarks safety

arXiv Machine Learning

Jun 30

BrainJanus: A Unified Model for Understanding and Generation across Brain, Vision, and Language

arXiv:2606. 30319v1 Announce Type: cross Abstract: Modeling the bidirectional correspondence between external sensory stimuli and internal neural activity has emerged as a critical frontier in neuroscience.

By Haitao Wu, Qirui Zhang, Zhouheng Yao, Shangquan Sun, Qihao Zheng, Mianxin Liu, Chi Zhang, Wanli Ouyang, Chunfeng Song, Changqing Zhang, Jiamin Wu

nlp multimodal benchmarks safety

arXiv AI

Jun 30

Evidence-Based Text-Conditioned 3D CT Synthesis for Ovarian Cancer

arXiv:2606. 28980v1 Announce Type: cross Abstract: Ovarian cancer is frequently diagnosed at an advanced stage, making preoperative contrast-enhanced computed tomography (CT) central to staging and surgical planning; yet the scarcity of annotated imaging data, compounded by privacy regulations, limits the development of generalizable computational models in this domain.

By Francesca Pia Panaccione, Eugenio Lomurno, Francesca Fati, Carlotta Pecchiari, Marina Rosanu, Luigi De Vitis, Lucia Ribero, Gabriella Schivardi, Giovanni Damiano Aletti, Nicoletta Colombo, Maria Francesca Spadea, Francesco Multinu, Matteo Matteucci, Elena De Momi

diffusion fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 30

Bridging the NISQ and Fault-Tolerant Regimes: Generative-ML-Assisted Quantum Selected CI for Molecular Simulations

arXiv:2606. 30551v1 Announce Type: cross Abstract: Calculation of binding energies for protein-ligand molecular systems requires accurate treatment of the electronic structure, a quantum chemistry problem that scales exponentially on classical hardware, while current quantum hardware remains too noisy for the required circuit depths.

By Anurag K. S. V., Ashish Kumar Patra, Manas Mukherjee, Ruchika Bhat, Sai Shankar P., Rahul Maitra, Jaiganesh G

benchmarks

arXiv Machine Learning

Jun 30

PCGD: Physics-Guided Conditional Graph Diffusion for TCAD Device Simulation

arXiv:2606. 29272v1 Announce Type: new Abstract: Technology computer-aided design (TCAD) semiconductor device simulation is fundamentally constrained by the high computational cost of iteratively solving coupled drift-diffusion equations.

By Yihan Zhang, Zhiteng Zhang, Kun Chen, Chen Wang

diffusion fine-tuning benchmarks

arXiv AI

Jun 30

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

arXiv:2606. 29955v1 Announce Type: cross Abstract: Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making.

By Jian Zhu, Yuzheng Zhang, Zeyao Ma, Bohan Zhang, Armin Schoepf, Daniel Woloch, Peter Yiliu Wang, Guangyu Robert Yang, Samuel Jacob, Siddharth Nagisetty, Abhiram Chundru, Jean Lin, Spencer Mateega, Jing Zhang

llms agents benchmarks

arXiv Machine Learning

Jun 30

BrainRiem: Riemannian Prototype Learning for Source-Free Cross-Site Brain Network Diagnosis

arXiv:2606. 29200v1 Announce Type: new Abstract: Multi-site functional MRI (fMRI) studies are essential for robust neuropsychiatric diagnosis yet suffer severe domain shifts from scanner heterogeneity, demographics, and site-specific acquisition protocols.

By Kunyu Zhang, Tianxiang Xu

fine-tuning benchmarks safety

arXiv AI

Jun 30

FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models

arXiv:2606. 29431v1 Announce Type: new Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image.

By Yichen Guo, Kai Tang, Fenglai Lin, Yiding Sun, Dongshuo Zhang, Wenya Wang, Lin William Cong, Shanghang Zhang

llms multimodal benchmarks safety

arXiv AI

Jun 30

Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation

arXiv:2606. 30190v1 Announce Type: cross Abstract: Existing domain-incremental learning (DIL) strategies call for massive amounts of data to adapt to new domains and suffer from the overfitting problem in the case of data scarcity.

By Naeem Paeedeh, Mahardhika Pratama, Wolfgang Mayer, Mukesh Prasad, Weiping Ding, Yew-Soon Ong

llms diffusion fine-tuning multimodal benchmarks

arXiv AI

Jun 30

Diagnosing and Mitigating Context Rot in Long-horizon Search

arXiv:2606. 29718v1 Announce Type: cross Abstract: Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon tasks.

By Shijie Xia, Yikun Wang, Zhen Huang, Pengfei Liu

llms efficiency benchmarks

arXiv Machine Learning

Jun 30

Solver-Integrated Adversarial Attacking and Training of Neural Operators

arXiv:2510. 18989v2 Announce Type: replace Abstract: Neural operators are commonly utilized as fast surrogates for numerical solvers in PDE problems, mapping input functions to solution functions.

By Yifei Sun

benchmarks safety

arXiv Machine Learning

Jun 30

CADS: Conformal Adaptive Decision System for Cost-Efficient Image Classification

arXiv:2605. 16401v2 Announce Type: replace-cross Abstract: While high-capacity AI models have advanced state-of-the-art performance, their practical deployment is often hindered by high inference costs, environmental impact, and a "one-size-fits-all" approach that ignores varying sample complexity.

By Mikael Turkoglu, Tim Bary, Vincent Thielens, Manon Dausort, Beno\^it Macq

computer-vision benchmarks

arXiv AI

Jun 30

Flow Matching in Feature Space for Stochastic World Modeling

arXiv:2606. 29059v1 Announce Type: cross Abstract: World modeling requires forecasting uncertain futures while preserving information useful for downstream perception.

By Francois Porcher, Nicolas Carion, Karteek Alahari, Shizhe Chen

diffusion multimodal benchmarks

arXiv AI

Jun 30

Dockerless: Environment-Free Program Verifier for Coding Agents

arXiv:2606. 28436v1 Announce Type: cross Abstract: Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning (RL).

By Wenhao Zeng, Yuling Shi, Xiaodong Gu, Chao Hu, Chaofan Wang, Yuhao Cui, Hongting Zhou, Mengnan Qi, Jianqiao Wangni, Zhaojian Yu, Shuzheng Gao, Kai Cai, Shilin He

agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

Building to the Test: Coding Agents Deliver What You Check, Not What You Requested

arXiv:2606. 28430v1 Announce Type: cross Abstract: Benchmarks are widely used to evaluate task completion by Large Language Models (LLMs), but this approach has accumulated construction-validity problems, and a passing score may not show whether the requested task was delivered.

By Yanuo Ma, Ben Kereopa-Yorke, Ben Schultz

llms agents benchmarks

arXiv AI

Jun 30

Multi-Agent DRL for QoS and Energy Optimization in RIS-Enabled Open-RAN Industrial 6G TN/NTN Networks

arXiv:2606. 28339v1 Announce Type: cross Abstract: Industrial 6G networks require ultra-reliable, low-latency, and energy-efficient connectivity in dynamic and blockage-prone environments, where conventional terrestrial deployments often fail to ensure stable coverage.

By Marwan Dhuheir, Thang X. Vu, Symeon Chatzinotas

agents reinforcement-learning benchmarks

Get the daily briefing