Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

17,067 stories · RSS feed

arXiv AI

Jul 8

VendorBench-100: A Unified Cross-Paradigm Benchmark for Deepfake Image Detection

arXiv:2607. 06254v1 Announce Type: cross Abstract: Deepfake image detection is currently served by three fundamentally different paradigms: commercial APIs, zero-shot vision-language models (LLMs), and open-source detectors.

By Sharayu N. Deshmukh, Md Rashidunnabi, Nelton Tiago Gemo, Kurundkar G. D., Mahamune M. R., Nilesh K. Deshmukh

llms diffusion multimodal benchmarks safety

arXiv AI

Jul 8

Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic SpeechLLMs

arXiv:2601. 12494v3 Announce Type: replace-cross Abstract: Audio large language models (LLMs) enable unified speech understanding and generation, but adapting them to linguistically complex and dialect-rich settings such as Arabic-English remains challenging.

By Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar Chowdhury

llms nlp fine-tuning multimodal benchmarks

arXiv AI

Jul 8

CMDR: Contextual Multimodal Document Retrieval

arXiv:2607. 05927v1 Announce Type: cross Abstract: Multimodal document retrieval aims to retrieve relevant pages while preserving both textual and visual content from the original document.

By Ryota Tanaka, Taku Hasegawa, Kyosuke Nishida

rag multimodal benchmarks

arXiv AI

Jul 8

Quantifying Frontier LLM Capabilities for Container Sandbox Escape

arXiv:2603. 02277v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks, creating novel security risks.

By Rahul Marchand, Art O Cathain, Jerome Wynne, Philippos Maximos Giavridis, Sam Deverett, John Wilkinson, Jason Gwartz, Harry Coppock

llms agents benchmarks safety

arXiv Machine Learning

Jul 8

Federated Physics-Grounded Reinforcement Learning for Distributed Stability Control in Smart Grids

arXiv:2607. 05553v1 Announce Type: new Abstract: Transient stability control in smart grids requires rapid post-fault damping of generator frequency and rotor angle deviations to prevent cascading failures.

By Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat, Eman Hammad

agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jul 8

Fine-Tuning Regimes Define Distinct Continual Learning Problems

arXiv:2604. 21927v3 Announce Type: replace Abstract: Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge.

By Paul-Tiberiu Iordache, Elena Burceanu

fine-tuning benchmarks

arXiv Machine Learning

Jul 8

Classification of Financial Data Using Quantum Support Vector Machine

arXiv:2412. 10860v2 Announce Type: replace-cross Abstract: Quantum Support Vector Machine is a kernel-based approach to classification problems.

By Seemanta Bhattacharjee, MD. Muhtasim Fuad, A. K. M. Fakhrul Hossain

benchmarks

arXiv AI

Jul 8

Universal Algorithm-Implicit Learning

arXiv:2602. 14761v2 Announce Type: replace-cross Abstract: Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability.

By Stefano Woerner, Seong Joon Oh, Christian F. Baumgartner

llms rag nlp multimodal benchmarks

arXiv AI

Jul 8

Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation

arXiv:2607. 05541v1 Announce Type: cross Abstract: Reinforcement Learning is commonly used to train large language models using environmental feedback.

By Muhammad Zain Amin, Kibele Sebnem Yildirim

llms reinforcement-learning efficiency benchmarks

arXiv AI

Jul 8

PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents

arXiv:2607. 06008v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external environments.

By Hongliang Li, Yijin Liu, Zhiwei Zhang, Zihe Liu, Xinyue Lou, Jinan Xu, Fandong Meng, Kaiyu Huang

llms agents benchmarks

arXiv AI

Jul 8

PatchOptic for Shared-State LLM Workflows with Projected Views and Verified Structured Updates

arXiv:2607. 05483v1 Announce Type: cross Abstract: Agentic workflows often operate over shared, structured state.

By Zhaoyu Bai, Jiaqi Cai

llms rag agents benchmarks

arXiv AI

Jul 8

x-Prediction Is All You Need:Training-Free Accelerated Generation via Endpoint Decodability

arXiv:2607. 06114v1 Announce Type: cross Abstract: Diffusion and flow matching models generate high-quality samples, but their ODE samplers often need tens to hundreds of neural function evaluations (NFEs).

By Xin Peng, Ang Gao

diffusion efficiency benchmarks

arXiv AI

Jul 8

FORGE: Towards Functional Tool-Use Generalization via Keypoint Trajectory Reasoning

arXiv:2607. 05780v1 Announce Type: cross Abstract: While humans readily repurpose a book, a stone, or a shoe to drive a nail, robots trained on specific tools fail to transfer the same function to novel ones -- a gap we formalize as functional generalization.

By Chuhao Zhou, Liquan Wang, Shuxin Cao, Xiangyu Chen, Yuxuan Hu, Boyu Ma, Animesh Garg, Jianfei Yang

robotics benchmarks

arXiv AI

Jul 8

SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigation

arXiv:2603. 21046v2 Announce Type: replace-cross Abstract: UAVs play an important role in applications such as autonomous exploration, disaster response, and infrastructure inspection.

By Wen Jiang, Kangyao Huang, Li Wang, Wang Xu, Wei Fan, Jinyuan Liu, Shaoyu Liu, Hanfang Liang, Hongwei Duan, Bin Xu, Xiangyang Ji, Huaping Liu

agents multimodal benchmarks safety

arXiv Machine Learning

Jul 8

PoseVLA: Universal Pose Pretraining for Generalizable Vision-Language-Action Policies

arXiv:2602. 19710v3 Announce Type: replace-cross Abstract: Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level perception with sparse, embodiment-specific action supervision.

By Haitao Lin, Hanyang Yu, Jingshun Huang, He Zhang, Yonggen Ling, Ping Tan, Xiangyang Xue, Yanwei Fu

nlp robotics multimodal benchmarks safety

arXiv Machine Learning

Jul 8

Intuitionistic Fuzzy Graph Embedded Random Vector Functional Link with Multiview Learning

arXiv:2607. 05635v1 Announce Type: new Abstract: Random Vector Functional Link (RVFL) networks are popular due to their fast training and universal approximation capabilities.

By Vrushank Ahire, Yogesh Kumar, M. A. Ganaie

rag benchmarks

arXiv Machine Learning

Jul 8

Low-Overhead Error-Corrected QCNNs Using Bivariate Bicycle Codes

arXiv:2607. 05724v1 Announce Type: new Abstract: Quantum convolutional neural networks (QCNNs) combine the power of quantum computing and classical CNN for computational speedup in classification tasks.

By Alejandro Rosales, Animesh Yadav

benchmarks

arXiv AI

Jul 8

SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation

arXiv:2607. 05943v1 Announce Type: new Abstract: Training multimodal search agents to perform multi-hop reasoning remains challenging due to a fundamental structural disconnect: existing pipelines construct training data, search environments, and reward signals independently, causing synthesized structural metadata to be discarded, environments to rely on irreproducible external engines, and RL rewards to remain sparse at the trajectory level.

By Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Jiapeng li, Qunzhong Wang, Tailai Chen, Qianshan Wei, Chuan Xiao, Shanyu Rong, Yangfu Li, Yanhan Zhou, Yunpu Ma, Yifan Zhang, Xiangyu Yue

agents reinforcement-learning multimodal benchmarks

arXiv AI

Jul 8

FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games

arXiv:2607. 06514v1 Announce Type: new Abstract: We present FootsiesGym, an open-source environment for learning in a non-trivial two-player, zero-sum, imperfect-information game.

By Chase McDonald, Nathan Tsang, Wesley N. Kerr

reinforcement-learning efficiency benchmarks

arXiv AI

Jul 8

FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

arXiv:2607. 06519v1 Announce Type: new Abstract: Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning.

By Anna C\'ordoba, Adam Puente Tercero, Nerea Angulo Hijo, Mar Linares Tercero, Julia Barrientos, Ainhoa Miranda, Jes\'us Olivera

llms nlp efficiency benchmarks

Get the daily briefing