Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,767 stories · RSS feed

arXiv Machine Learning

Jun 25

Two-dimensional Hyperbolic RNN Neural Quantum State

arXiv:2606. 25600v1 Announce Type: cross Abstract: In the first part of this work, we construct the first type of two-dimensional (2D) hyperbolic neural quantum state (NQS) in the form of the Lorentz 2DRNN (Recurrent Neural Network) and benchmark its performance against the Euclidean 2DRNN in the paradigmatic $N\times N$ 2D Transverse Field Ising Model (2DTFIM) setting with different lattice sizes up to $N=12$ and at different transverse magnetic field strengths.

By H. L. Dao

benchmarks

arXiv Machine Learning

Jun 25

InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy

arXiv:2606. 25984v1 Announce Type: cross Abstract: Large language models are increasingly deployed as investment research assistants, yet no benchmark tests whether they can accurately reconstruct and apply the specific procedural decision frameworks of expert investors.

By Mingguang Chen, Bo Qu

llms benchmarks

arXiv Machine Learning

Jun 25

What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics

arXiv:2606. 25182v1 Announce Type: cross Abstract: Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training.

By Sofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal

llms benchmarks safety

arXiv Machine Learning

Jun 25

Latent Block-Diffusion Temporal Point Processes: A Semi-Autoregressive Framework for Asynchronous Event Sequence Generation

arXiv:2606. 24982v1 Announce Type: new Abstract: Modeling and sampling from the underlying distribution of asynchronous event sequences are crucial in various real-world applications, including social networks, medical diagnosis, and financial transactions.

By Shuai Zhang, Yancheng Chen, Chuan Zhou, Yang Liu, Xixun Lin, Xiangyu Zhao, Jun Zhu, Zhi-Ming Ma

diffusion benchmarks

arXiv Machine Learning

Jun 25

RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments

arXiv:2606. 26094v1 Announce Type: new Abstract: For most of scientific history, researchers studying behavior could only infer hidden mechanisms from outward actions: an inverse problem that becomes more tractable when observation is augmented by targeted intervention.

By Babak Rahmani, Sebastian Dziadzio, Joschka Str\"uber, Sergio Hern\'andez-Guti\'errez, Matthias Bethge

llms agents benchmarks safety

arXiv Machine Learning

Jun 25

A Hybrid CNN-LSTM Intrusion Detection Framework for Cybersecurity in Smart Renewable Energy Grids

arXiv:2606. 25200v1 Announce Type: new Abstract: The accelerated digitalization of renewable energy smart grids through IoT sensors, AMI, and SCADA systems has significantly expanded the attack surface for sophisticated cyberattacks, FDI attacks that stealthily distort state estimation and DoS/DDoS attacks that flood communication channels.

By Sajib Debnath, Remon Das

efficiency benchmarks

arXiv Machine Learning

Jun 25

Onsager-Machlup Posterior Transport for Deep Gaussian Processes

arXiv:2605. 23434v2 Announce Type: replace Abstract: Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs).

By Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

diffusion benchmarks

arXiv Machine Learning

Jun 25

The Inference-Compute Frontier and a Latency-Efficient Architecture for Limit Order Book Prediction

arXiv:2606. 25986v1 Announce Type: new Abstract: We study whether a scaling-law-style inference-compute frontier appears in limit order book prediction.

By C. Evans Hedges

benchmarks

arXiv Machine Learning

Jun 25

TopoCast: A Topological Fidelity Framework for Evaluating Transformer-Based Time Series Forecasting

arXiv:2606. 25439v1 Announce Type: new Abstract: Deep learning-based models have achieved state-of-the-art performance in Time Series Forecasting (TSF), yet their evaluation remains dominated by pointwise error metrics such as Mean Squared Error (MSE), which quantify numerical accuracy but overlook structural properties of the forecast signal, including recurrent dynamics, oscillatory behavior, and phase alignment.

By Sandeepa Weerasekara, Sandareka Wickramanayake

llms rag benchmarks safety

arXiv Machine Learning

Jun 25

Hierarchical Graph Learning for Calendar Spread Strategies in Commodity Futures Markets

arXiv:2606. 25811v1 Announce Type: cross Abstract: Commodity futures can be represented hierarchically, with underlying assets at the upper level and individual futures contracts at the lower level.

By Yoonsik Hong, Diego Klabjan

benchmarks

arXiv Machine Learning

Jun 25

Swarm-Inspired Generation of Collective Behaviors in Graph Dynamical Systems

arXiv:2606. 24958v1 Announce Type: new Abstract: Collective behavior arises when locally interacting units produce coordinated global organization, from synchronization in dynamical systems to task-relevant information flow on graphs.

By Ji Chen, Song Chen, Chengzhang Gong, Li Fan, Chao Xu

agents robotics benchmarks

arXiv Machine Learning

Jun 25

MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning

arXiv:2606. 25225v1 Announce Type: cross Abstract: Self-supervised learning from large-scale video data has emerged as a dominant paradigm for visual representation learning.

By Revant Teotia, Adrien Bardes, Michael Rabbat, Sumit Chopra, Matthew J. Muckley, Nicolas Ballas

rag multimodal benchmarks

arXiv Machine Learning

Jun 25

LLM Performance on a Real, Double-Marked GCSE Benchmark

arXiv:2606. 24973v1 Announce Type: cross Abstract: We introduce a dataset of 32,534 double-marked real student responses to GCSE mock exams (GCSEs are the UK's national exams, taken at age ~16), spanning 328 questions across five subjects and including handwritten work.

By Malachy Fox, Kavi Samra, Paul Jung

llms benchmarks

arXiv Machine Learning

Jun 25

The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

arXiv:2606. 25450v1 Announce Type: new Abstract: Traditional evaluations measure a learning algorithm's final performance on an i.

By Jinghan Zhang, Zerui Cheng, Shiqi Chen, Ge Zhang, Wenhao Huang, Jiashuo Liu, Junxian He, Tianle Cai

reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 25

Multi-Stream Temporal Fusion for Financial Fraud Detection

arXiv:2606. 25007v1 Announce Type: new Abstract: Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams -- transactions, login sessions, risk signals -- that individually appear benign but collectively reveal fraudulent patterns.

By Mohammadamin Dashti Moghaddam, Nick Sciarrilli

llms benchmarks safety

arXiv Machine Learning

Jun 25

Do Thinking Tokens Help with Safety?

arXiv:2606. 25013v1 Announce Type: new Abstract: Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.

By Narutatsu Ri, Abhishek Panigrahi, Sanjeev Arora

llms benchmarks safety

arXiv Machine Learning

Jun 25

RotRNN: Modelling Long Sequences with Rotations

arXiv:2407. 07239v3 Announce Type: replace Abstract: Linear recurrent neural networks, such as State Space Models (SSMs) and Linear Recurrent Units (LRUs), have recently shown state-of-the-art performance on long sequence modelling benchmarks.

By Kai Biegun, Rares Dolga, Jake Cunningham, David Barber

benchmarks

arXiv Machine Learning

Jun 25

ConSolv: Solvent-Conditional Machine Learning Implicit Solvent Potential

arXiv:2606. 24983v1 Announce Type: cross Abstract: Implicit solvent machine learning potentials (MLPs) offer a powerful route to bridging the gap between accuracy and efficiency in molecular simulations.

By Linying Zhang, Julija Zavadlav

rag benchmarks

arXiv Machine Learning

Jun 25

Interpretable Concept-Guided Polynomial Tabular Kolmogorov-Arnold Network for EEG-Based Mild Cognitive Impairment Detection

arXiv:2606. 25434v1 Announce Type: new Abstract: Early and scalable detection of mild cognitive impairment (MCI) remains an unresolved clinical challenge.

By Yosef Bernardus Wirian, Qiang Cheng

benchmarks safety

arXiv Machine Learning

Jun 25

Forget to Improve: On-Device LLM-Agent Continual Learning via Budget-Curated Memory

arXiv:2606. 25115v1 Announce Type: new Abstract: On-device language-model agents improve by accumulating experience in retrieved memory rather than by updating weights.

By Beining Wu, Zihao Ding, Jun Huang, Yanxiao Zhao

llms agents robotics benchmarks

Get the daily briefing