Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,305 stories · RSS feed

arXiv Machine Learning

Jun 3

SAHG: Sector-Anisotropic Hyperbolic Graph Model for Social Bot Detection

arXiv:2605. 30166v2 Announce Type: replace-cross Abstract: LLM-driven social bots can generate fluent, human-like text, reducing the discriminative advantage of content-based detection alone.

By Hanning Lu, Yingguang Yang, Jinwei Su, Yang Liu, Zhaoqian Yao, Yaoming Li, Taoran Liang, Ziyi Zhang, Ran Ran, Kefu Xu, Bin Chong

llms benchmarks safety

arXiv Machine Learning

Jun 3

WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents

arXiv:2605. 20306v2 Announce Type: replace-cross Abstract: We introduce WildRoadBench, a wild aerial road-damage grounding benchmark that couples direct visual grounding by vision-language models with autonomous research-and-engineering by LLM-driven agents on a single professionally annotated UAV corpus.

By Bingnan Liu, Chenhang Cui, Rui Huang, Jiani Luo, Zhirong Shen, Tinghao Wang, Xiande Huang, Lingbei Meng, Fei Shen, An Zhang

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 3

LC-SAC: Lyapunov-Constrained Soft Actor-Critic via Koopman Operator Theory for Trajectory Tracking and Stabilization

arXiv:2602. 04132v4 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) has achieved remarkable success in solving complex sequential decision-making problems.

By Dhruv S. Kushwaha, Zoleikha A. Biron

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 3

Towards Blind Lens Aberration Correction via Large LensLib Pre-training and Discrete Degradation Priors

arXiv:2511. 17126v4 Announce Type: replace-cross Abstract: Emerging deep-learning-based lens library pre-training (LensLib-PT) pipeline offers a new avenue for blind lens aberration correction by training a universal neural network, demonstrating strong capability in handling diverse unknown optical degradations.

By Xiaolong Qian, Qi Jiang, Yao Gao, Lei Sun, Kailun Yang, Xian Wang, Zhonghua Yi, Wenyong Li, Ming-Hsuan Yang, Luc Van Gool, Kaiwei Wang

efficiency benchmarks

arXiv AI

Jun 3

CoMPAS3D: A Dataset and Benchmark for Interactive Motion

arXiv:2507. 19684v2 Announce Type: replace-cross Abstract: Socially interactive humanoid robots must engage with humans through their bodies, adapting in real time to a partner's movement, intent, and abilities.

By Bermet Burkanova, Yasaman Etesam, Payam Jome Yazdian, Trinity Evans, Chuxuan Zhang, Zoe Stanley, Paige Tutt\"os\'i, Angelica Lim

llms robotics fine-tuning multimodal benchmarks safety

arXiv AI

Jun 3

UR$^2$: Unify RAG and Reasoning through Reinforcement Learning

arXiv:2508. 06165v5 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have shown strong capabilities through two complementary paradigms: Retrieval-Augmented Generation (RAG) for knowledge grounding and Reinforcement Learning from Verifiable Rewards (RLVR) for complex reasoning.

By Weitao Li, Boran Xiang, Xiaolong Wang, Zhinan Gou, Weizhi Ma, Yang Liu

llms rag reinforcement-learning benchmarks

arXiv Machine Learning

Jun 3

Reading the Finetuning Prior: Verbatim Content Recovery via Contrastive Decoding Diffing

arXiv:2605. 25902v2 Announce Type: replace Abstract: Narrowly finetuned language models memorize implanted content verbatim, but auditing what a deployed model has been taught, without access to its weights or training data, remains an open challenge.

By Micha{\l} Brzozowski, Zuzanna Dubanowska, Enrico Cassano, Neo Christopher Chung

llms fine-tuning benchmarks data-engineering

arXiv Machine Learning

Jun 3

Parallel Adaptive Multi-Objective Evolutionary Learning of Discretized Bayesian Network Classifiers for Clinical Data

arXiv:2605. 29058v2 Announce Type: replace Abstract: Bayesian Networks (BNs) are of interest from an explainable AI viewpoint, offering transparent probabilistic models for decision support.

By Damy M. F. Ha, Thalea Schlender, Yvette M. van der Linden, Peter A. N. Bosman, Tanja Alderliesten

benchmarks

arXiv AI

Jun 3

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining

arXiv:2509. 22468v2 Announce Type: replace-cross Abstract: High-quality molecular representations are essential for property prediction and molecular design, yet large labeled datasets remain scarce.

By Boshra Ariguib, Mathias Niepert, Andrei Manolache

llms rag diffusion fine-tuning multimodal benchmarks

arXiv AI

Jun 3

ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models

arXiv:2510. 09711v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches.

By Wenbin Guo, Xin Wang, Jiaoyan Chen, Lingbing Guo, Zhao Li, Zirui Chen

llms rag nlp efficiency benchmarks

arXiv Machine Learning

Jun 3

ReciNet: Reciprocal Space-Aware Long-Range Modeling for Crystalline Property Prediction

arXiv:2502. 02748v4 Announce Type: replace Abstract: Predicting properties of crystals from their structures is a fundamental yet challenging task in materials science.

By Jianan Nie, Peiyao Xiao, Kaiyi Ji, Peng Gao

benchmarks

arXiv Machine Learning

Jun 3

Outsmarting the Chameleon: Counterfactual Decoupling for Tactical OOD Shifts in Live Streaming Risk Assessment

arXiv:2606. 02946v1 Announce Type: new Abstract: Live streaming has emerged as a primary medium for social interaction and digital commerce, yet it is increasingly plagued by sophisticated risks.

By Yiran Qiao, Jing Chen, Jiaqi Xu, Yang Liu, Qiwei Zhong, Xiang Ao

benchmarks safety

arXiv AI

Jun 3

PHASE: Physiology-Aware Hyperspectral Reconstruction via Object-to-Human Domain Adaptation

arXiv:2511. 13020v2 Announce Type: replace-cross Abstract: Although hyperspectral imaging offers unparalleled non-invasive physiological insight, its bulky hardware, slow acquisition, and regulatory burden severely limit its clinical availability.

By Yufei Wen, Shuxing Zhong, Jingdan Kang, Yuting Zhang, Jintai Chen, Kaishun Wu

fine-tuning benchmarks safety

arXiv Machine Learning

Jun 3

Forecasting Conceptual Diffusion in Science: The Case of Quantum Computing

arXiv:2606. 03919v1 Announce Type: cross Abstract: Understanding and anticipating scientific change requires models that distinguish between endogenous consolidation and exogenous diffusion of scientific concepts.

By Thomas Maillart, Thibaut Chataing, David Dosu, Paul Bagourd, Julian Jang-Jaccard, Alain Mermoud

diffusion robotics benchmarks

arXiv Machine Learning

Jun 3

Visual Instruction Tuning Aligns Modalities through Abstraction

arXiv:2606. 03871v1 Announce Type: cross Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text.

By Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 3

Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

arXiv:2606. 03988v1 Announce Type: new Abstract: Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable.

By Mahtab Bigverdi, Lindsey Li, Weikai Huang, Yiming Liu, Jaemin Cho, Jieyu Zhang, Tuhin Kundu, Chris Dangjoo Kim, Zelun Luo, Linda Shapiro, Ranjay Krishna

llms multimodal benchmarks

arXiv AI

Jun 3

LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks

arXiv:2606. 03303v1 Announce Type: new Abstract: Large Language Models (LLMs) exhibit strong informal mathematical reasoning but struggle to generate mechanically verifiable proofs in formal languages like Lean.

By Po-Nien Kung, Linfeng Song, Dawsen Hwang, Jinsung Yoon, Chun-Liang Li, Simone Severini, Mirek Ol\v{s}\'ak, Edward Lockhart, Quoc V Le, Burak Gokturk, Thang Luong, Tomas Pfister, Nanyun Peng

llms agents benchmarks

arXiv AI

Jun 3

VidMsg: A Benchmark for Implicit Message Inference in Short Videos

arXiv:2606. 03635v1 Announce Type: cross Abstract: Understanding short online videos involves more than identifying visible objects and actions; video makers often include an underlying message or purpose in the clip.

By Issar Tzachor, Michael Green, Rami Ben-Ari

llms benchmarks safety

arXiv AI

Jun 3

Toward Training Superintelligent Software Agents through Self-Play SWE-RL

arXiv:2512. 18552v3 Announce Type: replace-cross Abstract: While current software agents powered by large language models (LLMs) and agentic reinforcement learning (RL) can boost programmer productivity, their training data (e.

By Yuxiang Wei, Zhiqing Sun, Emily McMilin, Jonas Gehring, David Zhang, Gabriel Synnaeve, Daniel Fried, Lingming Zhang, Sida Wang

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 3

Black-box, Adaptive, Efficient, Transferable, Harmful, Applicable... Attacks Are All You Need to Break LLMs

arXiv:2606. 03647v1 Announce Type: cross Abstract: Accurately evaluating adversarial robustness is a longstanding challenge.

By Vincent Limbach, Jonas Dornbusch, David L\"udke, Stephan G\"unnemann, Leo Schwinn

llms diffusion fine-tuning benchmarks safety

Get the daily briefing