Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv AI

Oct 27

Agentic AI Empowered Intent-Based Networking for 6G

arXiv:2601.06640v2 Announce Type: replace Abstract: The transition towards sixth-generation (6G) wireless networks necessitates autonomous orchestration mechanisms capable of translating high-level o...

By Genze Jiang, Kezhi Wang, Xiaomin Chen, Yizhou Huang

llms agents benchmarks safety

arXiv AI

Oct 27

Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs

arXiv:2602.08241v2 Announce Type: replace Abstract: While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing app...

By Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao, Xuelong Li

llms reinforcement-learning multimodal benchmarks

arXiv AI

Oct 27

AlphaCNOT: Learning CNOT Minimization with Model-Based Planning

arXiv:2604.13812v2 Announce Type: replace Abstract: Quantum circuit optimization is a central task in quantum computing, as current NISQ devices suffer from error propagation that often scales with t...

By Jacopo Cossio, Daniele Lizzio Bosco, Riccardo Romanello, Giuseppe Serra, Carla Piazza

reinforcement-learning benchmarks

arXiv AI

Oct 27

NGM: A Plug-and-Play Training-Free Memory Module for LLMs

arXiv:2605.16893v2 Announce Type: replace Abstract: Recent studies have introduced memory-augmented architectures that decouple knowledge storage from neural computation, enabling more direct access...

By Yuwen Qu, Wenhui Dong, Camil Hamami, Ming Zhao, Chenyang Si, Caifeng Shan

llms rag multimodal benchmarks

arXiv AI

Oct 27

XPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding

arXiv:2608.02438v2 Announce Type: replace Abstract: Block-diffusion drafters like dFlash generate an entire block of draft tokens in a single forward pass, drastically reducing the overhead of multip...

By Zheng Wang, Davis Wertheimer, Yu Chin Fabian Lim, Mudhakar Srivatsa, Raghu K. Ganti, Minjia Zhang, Naigang Wang

diffusion efficiency benchmarks

arXiv AI

Oct 27

DISCERN: Can AI Agents Work Like Scientists and Guide Discovery?

arXiv:2609.33357v2 Announce Type: replace Abstract: Reliable automated research requires agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially red...

By Nan Huang, Mario Tapia-Pacheco, Kun Zhou, Yiming Huang, Kevin Jos\'e Barrientos D\'iaz, Tiffany Amariuta, Jingbo Shang

agents benchmarks safety

arXiv AI

Oct 27

MoveBench: A Rubric-Based Benchmark for Evaluating Large Language Models in Sports and Health Coaching

arXiv:2610.26185v2 Announce Type: replace Abstract: Despite rapid advances in large language models (LLMs), their deployment in the sports and health domain remains high-risk due to the absence of do...

By Yitong Li, Jiang Bian, Jingling Wang, Pan Ding, Chengying Gu

llms nlp benchmarks safety

arXiv AI

Oct 27

RIDAR: Safeguarding Research Integrity in Autonomous Research Agents via Automated Detection and Runtime Intervention

arXiv:2610.31372v2 Announce Type: replace Abstract: Autonomous research agents are increasingly used to automate large portions of scientific research, yet they are prone to research integrity violat...

By Jiening Siow, Amy Xin, Junjie Wang, Ziyu Tang, Zijun Yao, Yunjia Qi, Fanjin Zhang, Lei Hou, Jie Tang, Juanzi Li

agents benchmarks

arXiv AI

Oct 27

Do Deepfake Detectors Work in Reality?

arXiv:2502.10920v2 Announce Type: replace-cross Abstract: Deepfakes, particularly those involving faceswap-based manipulations, have sparked significant societal concern due to their increasing reali...

By Simiao Ren, Hengwei Xu, Tsang Ng, Kidus Zewde, Shengkai Jiang, Ramini Desai, Disha Patil, Ning-Yau Cheng, Yining Zhou, Ragavi Muthukrishnan

benchmarks

arXiv AI

Oct 27

Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception

arXiv:2508.11256v2 Announce Type: replace-cross Abstract: Dense visual perception tasks have been constrained by their reliance on predefined categories, limiting their applicability in real-world sc...

By Junjie Wang, Keyu Chen, Yulin Li, Bin Chen, Hengshuang Zhao, Xiaojuan Qi, Zhuotao Tian

llms diffusion computer-vision multimodal benchmarks

arXiv AI

Oct 27

Simple yet Effective Semi-supervised Knowledge Distillation from Vision-Language Models via Dual-Head Optimization

arXiv:2505.07675v3 Announce Type: replace-cross Abstract: Semi-supervised learning (SSL) has emerged as a practical solution for addressing data scarcity challenges by leveraging unlabeled data. Rece...

By Seongjae Kang, Dong Bok Lee, Hyungjoon Jang, Sung Ju Hwang

llms efficiency multimodal benchmarks

arXiv AI

Oct 27

CRAFT-E: An Interpretable and Editable Neuro-Symbolic Framework for Affordance-Grounded Active Object Retrieval

arXiv:2512.04231v2 Announce Type: replace-cross Abstract: Assistive robots operating in unstructured environments must reason not only about what objects are, but how they can be used, including non-...

By Zhou Chen, Joe Lin, Carson Bulgin, Sathyanarayanan N. Aakur

llms computer-vision robotics benchmarks safety

arXiv AI

Oct 27

Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought

arXiv:2512.21711v2 Announce Type: replace-cross Abstract: Latent tokens are gaining attention for enhancing reasoning in large language models (LLMs), yet their internal mechanisms remain unclear. Th...

By Yuyi Zhang, Boyu Tang, Tianjie Ju, Sufeng Duan, Gongshen Liu

llms benchmarks safety

arXiv AI

Oct 27

Vulnerability Analysis for Safe Multi-Agent Reinforcement Learning

arXiv:2610.27293v2 Announce Type: replace Abstract: Safe Multi-Agent Reinforcement Learning (SMARL) aims to learn cooperative control policies under prescribed safety constraints. Al- though existing...

By Taozheng Zhu, Shixiong Jiang, Fanxin Kong

agents reinforcement-learning benchmarks safety

arXiv AI

Oct 27

FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation

arXiv:2602.13444v2 Announce Type: replace-cross Abstract: Recent vision-language-action (VLA) models can generate plausible end-effector motions, yet they often fail in long-horizon, contact-rich tas...

By Huajian Zeng, Lingyun Chen, Jiaqi Yang, Yuantai Zhang, Fan Shi, Peidong Liu, Xingxing Zuo

diffusion robotics multimodal benchmarks safety

arXiv AI

Oct 27

Flowette: Flow Matching with Graphette Priors for Graph Generation

arXiv:2602.23566v4 Announce Type: replace-cross Abstract: We study generative modeling of graphs with recurring subgraph motifs. We propose Flowette, a continuous flow matching framework that employs...

By Asiri Wijesinghe, Sevvandi Kandanaarachchi, Daniel M. Steinberg, Cheng Soon Ong

llms diffusion benchmarks safety

arXiv AI

Oct 27

EpiFlow: Bridging Safety and Performance in Offline Reinforcement Learning via Epigraph-Guided Flow Matching

arXiv:2602.08054v2 Announce Type: replace-cross Abstract: Offline reinforcement learning (RL) offers a promising framework for deploying autonomous systems in safety-critical settings without the ris...

By Manan Tayal, Mumuksh Tayal, Ravi Prakash

agents diffusion reinforcement-learning benchmarks safety

arXiv AI

Oct 27

Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTOR

arXiv:2603.06164v2 Announce Type: replace-cross Abstract: Self-supervised learning (SSL) underpins modern audio deepfake detection, yet most prior work centers on a single large wav2vec2-XLSR backbon...

By Ajinkya Kulkarni, Sandipana Dowerah, Atharva Kulkarni, Tanel Alum\"ae, Mathew Magimai Doss

llms benchmarks

arXiv AI

Oct 27

BatMIL: Geometry-Embedding State Space Model for Whole-Slide Image Representation

arXiv:2605.05164v2 Announce Type: replace-cross Abstract: Accurate analysis of histopathological images is critical for disease diagnosis and treatment planning. Whole-slide images (WSIs), which digi...

By Enhui Chai, Sicheng Chen, Tianyi Zhang, Chad Wong, Kecheng Huang, Zeyu Liu, Fei Xia

rag benchmarks

arXiv AI

Oct 27

OmicsLM: A Multimodal Large Language Model for Multi-Sample Omics Reasoning

arXiv:2605.06728v2 Announce Type: replace-cross Abstract: Large language models can reason over biological knowledge, but they are not naturally equipped to work with quantitative omics measurements....

By Maciej Sypetkowski, {\L}ukasz Smoli{\'n}ski, Remigiusz Kinas, Joanna Krawczyk, Bart{\l}omiej Hofman, Aleksandra Mo{\.z}wi{\l}{\l}o, Micha{\l} Pikusa, Tomasz Jetka, Rafa{\l} Powalski

llms rag fine-tuning multimodal benchmarks

Get the daily briefing