Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,561 stories · RSS feed

arXiv AI

Jun 15

Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

arXiv:2606. 13994v1 Announce Type: cross Abstract: LLM-based Agents are becoming increasingly capable and widely deployed, creating growing incentives for adversarial misuse in the real-world.

By Vikhyath Kothamasu, Virginia Smith, Chhavi Yadav

llms agents benchmarks safety

arXiv Machine Learning

Jun 15

Where Black-box Drug-Target Interaction Prediction Models Look: Cross-Method Explainability

arXiv:2606. 14245v1 Announce Type: new Abstract: Drug-target interaction (DTI) and affinity (DTA) predictors increasingly achieve strong benchmark scores, yet their internal use of sequence, fingerprint, and graph features often remains opaque.

By Ali Vefghi, Zahed Rahmati, Mohammad Akbari

benchmarks safety

arXiv AI

Jun 15

Relational Retrieval: Leveraging Known-Novel Interactions for Generalized Category Discovery

arXiv:2605. 09420v2 Announce Type: replace-cross Abstract: In this study, we tackle Generalized Category Discovery (GCD) via a Relational Retrieval perspective, explicitly coupling labeled and unlabeled data through bidirectional knowledge transfer.

By Yulin Xu, Chunqi Guo, Yuanzhen Shuai, Jianyuan Ni

benchmarks safety

arXiv AI

Jun 15

ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning

arXiv:2606. 14697v1 Announce Type: cross Abstract: Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support.

By Sicheng Yang, Hangjie Yuan, Wenjun Zhang, Jinwang Wang, Yichen Qian, Weihua Chen, Fan Wang, Lei Zhu

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 15

Natively Unlearnable Large Language Models

arXiv:2606. 13873v1 Announce Type: new Abstract: Unlearning aims to remove the influence of specific training data sources, but this has proved challenging because the contributions of different sources are entangled within the model.

By Gaurav R. Ghosal, Pratyush Maini, Aditi Raghunathan

llms benchmarks safety

arXiv AI

Jun 15

ANSR-DT: A Neuro-Symbolic Framework for Adaptive and Explainable Digital Twins

arXiv:2501. 08561v4 Announce Type: replace Abstract: Digital twins are increasingly used to monitor and optimize industrial systems, yet many existing frameworks remain difficult to interpret, slow to adapt, and limited in their ability to incorporate explicit domain knowledge.

By Safayat Bin Hakim, Muhammad Adil, Alvaro Velasquez, Houbing Herbert Song

reinforcement-learning benchmarks safety

arXiv AI

Jun 15

MASLab: A Unified and Comprehensive Codebase for LLM-based Multi-Agent Systems

arXiv:2505. 16988v2 Announce Type: replace-cross Abstract: LLM-based multi-agent systems (MAS) have demonstrated significant potential in enhancing single LLMs to address complex and diverse tasks in practical applications.

By Rui Ye, Keduan Huang, Qimin Wu, Yuzhu Cai, Tian Jin, Xianghe Pang, Xiangrui Liu, Jiaqi Su, Chen Qian, Bohan Tang, Kaiqu Liang, Jiaao Chen, Yue Hu, Zhenfei Yin, Rongye Shi, Bo An, Yang Gao, Wenjun Wu, Lei Bai, Siheng Chen

llms agents benchmarks

arXiv AI

Jun 15

A Fixed-Point Neural Operator for Size- and Functional-Transferable Hamiltonian Prediction

arXiv:2606. 14498v1 Announce Type: cross Abstract: Predicting the Kohn-Sham Hamiltonian with machine learning can accelerate density functional theory while retaining access to molecular orbitals, energy levels, and electronic-structure observables that energy-only surrogates cannot resolve.

By Yunhong Lou, Xihang Yue, Xinran Wei, Tianqi Deng, Linchao Zhu

fine-tuning benchmarks

arXiv AI

Jun 15

UP-NRPA: User Portrait based Nested Rollout Policy Adaptation for Planning with Large Language Models in Goal-oriented Dialogue Systems

arXiv:2606. 13683v1 Announce Type: new Abstract: To address the challenge that current dialogue policy planning methods struggle to dynamically adapt to diverse user characteristics, this paper proposes a User Portrait based Nested Rollout Policy Adaptation (UP-NRPA) online framework with Large Language Models.

By Hui Wang, Fafa Zhang, Meng Liu, Xiangyu Chen, Chaoxu Mu

llms reinforcement-learning benchmarks

arXiv AI

Jun 15

Active Inference for Adaptive Traffic Signal Control in Noisy Nonstationary IoT Environments

arXiv:2606. 13698v1 Announce Type: cross Abstract: Urban traffic signal control at IoT-instrumented intersections must remain effective under sensor occlusion, weather attenuation, and nonstationary demand.

By D\'enes Toth, George Ambroladze, Edwin Sundberg, Ali Beikmohammadi, Alfreds Lapkovskis

benchmarks

arXiv Machine Learning

Jun 15

Curvature-Guided Geometric Representation for Protein-Ligand Binding Affinity Prediction

arXiv:2606. 14159v1 Announce Type: new Abstract: Protein-ligand binding affinity (PLA) prediction is critical in drug discovery.

By Shuai Li, Chuan-Xian Ren, Yuhao Li, Ziqi Huang, Yue Pan, Mingzhe Tang, Hong Yan

benchmarks safety

arXiv Machine Learning

Jun 15

Decoupled Latent Optimization of Diffusion Models for Full Waveform Inversion

arXiv:2606. 14139v1 Announce Type: new Abstract: Full waveform inversion (FWI) recovers subsurface velocity from seismic recordings by solving a severely ill-posed, nonconvex PDE-constrained optimization.

By Chen Min, Zheng Ma

diffusion benchmarks

arXiv AI

Jun 15

Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders

arXiv:2503. 19947v2 Announce Type: replace-cross Abstract: Generalized metric depth understanding is critical for precise vision-guided robotics, which current state-of-the-art (SOTA) vision-encoders do not support.

By Paul Koch, J\"org Kr\"uger

diffusion computer-vision robotics fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 15

NEST3D: A High-Resolution Multimodal Dataset of Sociable Weaver Tree Nests

arXiv:2606. 14562v1 Announce Type: cross Abstract: Sociable weaver nests function as complex ecological structures offering thermoregulatory microhabitats and sustaining diverse species; however, datasets used in prior studies lack fine-grained 3D structural detail.

By Constanza A. Molina Catricheo, Simon Boeder, Ting-Jia Guo, Giacomo May, Cl\'ement Berthelot, Devis Tuia, Friedrich Fedor Reinhard, Fabio Remondino, Benjamin Risse

llms computer-vision robotics multimodal benchmarks

arXiv Machine Learning

Jun 15

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

arXiv:2606. 14299v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) such as CLIP have become a standard backbone for open-vocabulary recognition, yet their zero-shot predictions remain vulnerable to distribution shifts encountered at deployment.

By Jiazhen Huang, Xiao Chen, Zhiming Liu, Yaru Sun, Jingyan Jiang, Zhi Wang

llms multimodal benchmarks

arXiv Machine Learning

Jun 15

Arbitrary control over multimode wave propagation for machine learning

arXiv:2402. 17750v2 Announce Type: replace-cross Abstract: Controlled multimode wave propagation can enable more space-efficient photonic processors than architectures based on discrete components connected by single-mode waveguides.

By Tatsuhiro Onodera, Martin M. Stein, Benjamin A. Ash, Mandar M. Sohoni, Melissa Bosch, Ryotatsu Yanagimoto, Marc Jankowski, Timothy P. McKenna, Tianyu Wang, Gennady Shvets, Maxim R. Shcherbakov, Logan G. Wright, Peter L. McMahon

benchmarks

arXiv Machine Learning

Jun 15

Temporally Consistent Graph Q-Networks for Intelligent Network Control

arXiv:2606. 13848v1 Announce Type: cross Abstract: Mobile networks continue to grow in complexity and next generation networks are expected to support both increasing traffic loads and more diverse services.

By Zacharias Veiksaar, Maxime Bouton

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 15

IntSeqBERT: Learning Arithmetic Structure in OEIS via Modulo-Spectrum Embeddings

arXiv:2603. 05556v2 Announce Type: replace Abstract: Integer sequences in the OEIS span values from single-digit constants to astronomical factorials and exponentials, making prediction challenging for standard tokenised models that cannot handle out-of-vocabulary values or exploit periodic arithmetic structure.

By Kazuhisa Nakasho

llms rag benchmarks

arXiv AI

Jun 15

StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance

arXiv:2606. 14571v1 Announce Type: new Abstract: A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance.

By Guanming Liu, Yuqi Ren, Hansu Gu, Peng Zhang, Weihang Wang, Jiahao Liu, Ning Gu, Tun Lu

agents benchmarks

arXiv Machine Learning

Jun 15

CANN-EUCLID: unsupervised constitutive artificial neural network model discovery from full-field data

arXiv:2606. 14565v1 Announce Type: cross Abstract: Constitutive artificial neural networks (CANNs) provide interpretable material model discovery, but have so far been used in stress-supervised settings based on apparent stress-strain data from homogeneous tests.

By Benjamin Alheit, Siddhant Kumar, Mathias Peirlinck

efficiency benchmarks

Get the daily briefing