Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv AI

Jul 13

Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation

arXiv:2607. 09600v1 Announce Type: new Abstract: Enhancing the reasoning capabilities of large language model (LLM) agents requires effective orchestration of diverse expert models and tools.

By Kaiji Zhou, Ales Leonardis, Yue Feng

llms agents benchmarks

arXiv Machine Learning

Jul 13

Federated Low-Rank Koopman Learning for Multivariate Time-Series Anomaly Detection in IoT Systems

arXiv:2607. 08978v1 Announce Type: new Abstract: Distributed IoT systems generate multivariate time-series streams for monitoring physical assets, servers, and embedded sensing platforms.

By Tung-Anh Nguyen, Van-Phuc Bui, Anh Tuyen Le, Kim Hue Ta, Minh Thuy Le, J. Andrew Zhang, Xiaojing Huang

efficiency benchmarks

arXiv AI

Jul 13

Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing

arXiv:2607. 09015v1 Announce Type: cross Abstract: We study contextual bandit problems with correlated arms and access to surrogate reward signals produced by a machine learning model, motivated by applications such as large language model (LLM) routing.

By Ajay Narayanan Sridhar, Ronak Singh, Mehrdad Mahdavi, Vijaykrishnan Narayanan

llms reinforcement-learning benchmarks

arXiv AI

Jul 13

Model Agnostic Graph Prompt Learning for Crystal Property Prediction

arXiv:2607. 08996v1 Announce Type: cross Abstract: Graph Neural Networks have emerged as a powerful tool for the fast and accurate prediction of various crystal properties.

By Shrimon Mukherjee, Kishalay Das, Partha Basuchowdhuri, Pawan Goyal, Niloy Ganguly

benchmarks

arXiv Machine Learning

Jul 13

Present but Rescaled: Chat-to-Agent Transfer of Additive Activation Steering

arXiv:2607. 09156v1 Announce Type: new Abstract: Additive activation steering (injecting a scaled residual-stream direction during generation) is calibrated almost entirely in single-turn chat, yet the models it targets are increasingly deployed as tool-using ReAct agents.

By Lucas Pinto

agents benchmarks safety

arXiv AI

Jul 13

All you need is SAMPAT

arXiv:2607. 09235v1 Announce Type: cross Abstract: The current state of the art in AI/ML rests on deep neural architectures, which, in general, suffer from a lack of interpretability.

By Jayadeva, Madhur Aswani

benchmarks safety

arXiv AI

Jul 13

LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms

arXiv:2607. 08791v1 Announce Type: cross Abstract: Designing effective multi-objective Bayesian optimization (MOBO) algorithms requires balancing many interdependent design choices whose optimal configuration is problem-dependent and typically demands deep expertise.

By Georgios Laskaris, Reuben Brasher, Niki van Stein, Elena Raponi, Thomas B\"ack, Florian Neukart

llms benchmarks

arXiv Machine Learning

Jul 13

COBS: Cumulant Order Block Sparse Attention

arXiv:2607. 09052v1 Announce Type: new Abstract: Block sparse attention is a hardware friendly way to alleviate the key-value (KV) cache read bottleneck in large language models (LLMs).

By Alexander Tian, Aditya Ghai, Sanjit Neelam, Zaal Vasania, Akshay Mishra

llms efficiency benchmarks

arXiv Machine Learning

Jul 13

Temporal Knowledge Graph Forecasting under Distribution Shifts: A Synthetic Evaluation

arXiv:2607. 09232v1 Announce Type: new Abstract: Temporal knowledge graphs (TKGs) represent evolving relational systems, whose underlying data-generating processes often change over time.

By Konrad \"Ozdemir, Julia Gastinger, Lukas Kirchdorfer, Heiner Stuckenschmidt

benchmarks

arXiv Machine Learning

Jul 13

CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

arXiv:2607. 09543v1 Announce Type: new Abstract: Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications.

By Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian, Yassir Bendou, Richard Gao

llms benchmarks

arXiv Machine Learning

Jul 13

Forking-Sequences: Statistically and Computationally Efficient Multi-Horizon Forecasting with Reduced Volatility

arXiv:2510. 04487v5 Announce Type: replace Abstract: While accuracy is a critical requirement for time series forecasting, an equally important desideratum is reasonable forecast volatility across forecast creation dates (FCDs).

By Willa Potosnak, Malcolm Wolff, Mengfei Cao, Ruijun Ma, Tatiana Konstantinova, Dmitry Efimov, Michael W. Mahoney, Boris Oreshkin, Kin G. Olivares

llms benchmarks

arXiv AI

Jul 13

ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning

arXiv:2512. 16861v2 Announce Type: replace-cross Abstract: Long-horizon manipulation has been a long-standing challenge in the robotics community.

By Zihan Zhou, Animesh Garg, Ajay Mandlekar, Caelan Garrett

reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jul 13

A Self-Evolving Agentic Framework for Metasurface Inverse Design

arXiv:2604. 01480v2 Announce Type: replace Abstract: Metasurface inverse design can realize complex optical functionality, but turning a target optical response into executable optimization code still requires substantial expertise in computational electromagnetics and solver-specific software engineering.

By Yi Huang, Bowen Zheng, Yunxi Dong, Hong Tang, Huan Zhao, S. M. Rakibul Hasan Shawon, Hualiang Zhang

agents benchmarks

arXiv AI

Jul 13

4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception

arXiv:2607. 09629v1 Announce Type: cross Abstract: Reliable autonomous driving requires full-scene perception that couples foreground objects with dense semantic layout.

By Xiaokai Bai, Lianqing Zheng, Runwei Guan, Songkai Wang, Siyuan Cao, Hui-liang Shen

agents robotics multimodal benchmarks

arXiv AI

Jul 13

A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions

arXiv:2607. 08776v1 Announce Type: cross Abstract: Despite the success of knowledge distillation (KD) in Large Language Models (LLMs), the underlying mechanism behind its efficacy remains unclear.

By Qingzhuo Wang, Ruiyang Qin, Zhenxin Qin, Wen Shen, Zhihua Wei

llms efficiency benchmarks

arXiv AI

Jul 13

SLIDERS: Systematic Reviews via Automated Evidence Synthesis and Reconciliation

arXiv:2604. 22294v2 Announce Type: replace-cross Abstract: Systematic reviews -- which requires comprehensive evidence collection and synthesis from large document corpora in response to targeted research questions -- are foundational in finance, social sciences, and other technical fields.

By Harshit Joshi, Priyank Shethia, Jadelynn Dao, Monica S. Lam

llms rag agents benchmarks

arXiv Machine Learning

Jul 13

HALO: Hybrid Adaptive Latent Reasoning for Language Models

arXiv:2607. 08775v1 Announce Type: cross Abstract: We study how to improve a frozen pretrained language model with a small amount of adaptive extra computation.

By Micah Zhang

llms benchmarks

arXiv AI

Jul 13

Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026

arXiv:2607. 09623v1 Announce Type: cross Abstract: We present our submission to the QANTA 2026 shared challenge at the ICML 2026 Workshop on Efficient Multimodal Question Answering (EMM-QA).

By Nirjhar Das, Md. Al-Mamun Provath

llms agents nlp multimodal benchmarks

arXiv AI

Jul 13

DaDaDa: A Dataset for Data Pricing in Data Marketplaces

arXiv:2607. 08785v1 Announce Type: cross Abstract: High-quality data drives machine learning advances across industries.

By Qiheng Sun, Hongwei Zhang, Junxu Liu, Xiaokai Mao, Jinfei Liu, Kui Ren, Haibo Hu

benchmarks

arXiv AI

Jul 13

Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution

arXiv:2607. 08960v1 Announce Type: cross Abstract: Warehouse operations are governed by Standard Operating Procedures (SOPs) that encode complex, multi-system decision logic, which must be executed reliably under strict time constraints, yet LLM agents lack mechanisms to enforce procedural compliance and degrade under the context overload full SOP specifications introduce.

By Ning Liu, Kalle Kujanp\"a\"a, Zhaoxuan Zhu, P Aditya Sreekar, Kaiwen Liu, Chuanneng Sun, Jorge Marchena Menendez, Matthew Bales, Tianyu Yang, Shahnawaz Alam, Rose Yu, Baoyuan Liu, Kristina Klinkner, Shervin Malmasi

llms agents fine-tuning efficiency benchmarks

Get the daily briefing