Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,504 stories · RSS feed

arXiv Machine Learning

Jun 25

ConSolv: Solvent-Conditional Machine Learning Implicit Solvent Potential

arXiv:2606. 24983v1 Announce Type: cross Abstract: Implicit solvent machine learning potentials (MLPs) offer a powerful route to bridging the gap between accuracy and efficiency in molecular simulations.

By Linying Zhang, Julija Zavadlav

rag benchmarks

arXiv Machine Learning

Jun 25

RotRNN: Modelling Long Sequences with Rotations

arXiv:2407. 07239v3 Announce Type: replace Abstract: Linear recurrent neural networks, such as State Space Models (SSMs) and Linear Recurrent Units (LRUs), have recently shown state-of-the-art performance on long sequence modelling benchmarks.

By Kai Biegun, Rares Dolga, Jake Cunningham, David Barber

benchmarks

arXiv Machine Learning

Jun 25

Hierarchical Graph Learning for Calendar Spread Strategies in Commodity Futures Markets

arXiv:2606. 25811v1 Announce Type: cross Abstract: Commodity futures can be represented hierarchically, with underlying assets at the upper level and individual futures contracts at the lower level.

By Yoonsik Hong, Diego Klabjan

benchmarks

arXiv Machine Learning

Jun 25

Swarm-Inspired Generation of Collective Behaviors in Graph Dynamical Systems

arXiv:2606. 24958v1 Announce Type: new Abstract: Collective behavior arises when locally interacting units produce coordinated global organization, from synchronization in dynamical systems to task-relevant information flow on graphs.

By Ji Chen, Song Chen, Chengzhang Gong, Li Fan, Chao Xu

agents robotics benchmarks

arXiv Machine Learning

Jun 25

Pulmonary Embolism Risk Stratification from CTPA and Medical Records: Vascular Graphs Are Not All You Need

arXiv:2606. 25956v1 Announce Type: cross Abstract: Risk stratification for pulmonary embolism (PE) is critical for clinical decision-making.

By Nathan Painchaud, Tristan Hab\'emont, Morgane des Ligneris, Allan Serva, Pierre Croisille, Laurent Bertoletti, Thomas Lampert, Johannes F. Lutzeyer, Odyss\'ee Merveille

benchmarks

arXiv Machine Learning

Jun 25

Multi-Stream Temporal Fusion for Financial Fraud Detection

arXiv:2606. 25007v1 Announce Type: new Abstract: Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams -- transactions, login sessions, risk signals -- that individually appear benign but collectively reveal fraudulent patterns.

By Mohammadamin Dashti Moghaddam, Nick Sciarrilli

llms benchmarks safety

arXiv Machine Learning

Jun 25

Do Thinking Tokens Help with Safety?

arXiv:2606. 25013v1 Announce Type: new Abstract: Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.

By Narutatsu Ri, Abhishek Panigrahi, Sanjeev Arora

llms benchmarks safety

arXiv Machine Learning

Jun 25

Forget to Improve: On-Device LLM-Agent Continual Learning via Budget-Curated Memory

arXiv:2606. 25115v1 Announce Type: new Abstract: On-device language-model agents improve by accumulating experience in retrieved memory rather than by updating weights.

By Beining Wu, Zihao Ding, Jun Huang, Yanxiao Zhao

llms agents robotics benchmarks

arXiv Machine Learning

Jun 25

How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly Dissolves

arXiv:2606. 25092v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) models route each token to a few of many experts, inviting the hypothesis that experts form functional modules tied to capabilities or languages.

By Tony Salomone, Deep Gandhi, Ali Asaria

efficiency benchmarks

arXiv Machine Learning

Jun 25

Supervised Reinforcement Learning for the Coordination of Distributed Energy Resources

arXiv:2606. 24947v1 Announce Type: new Abstract: The increasing integration of distributed energy resources (DERs) is crucial for power system decarbonization, yet unlocking DERs' flexibility is challenged by their inherent uncertainties and modelling complexity.

By Haoyuan Deng, Yihong Zhou, Thomas Morstyn, Yi Wang

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 25

Model-agnostic Mitigation Strategies of Data Imbalance for Regression

arXiv:2506. 01486v2 Announce Type: replace Abstract: Data imbalance persists as a pervasive challenge in regression tasks, introducing bias in model performance and undermining predictive reliability.

By Jelke Wibbeke, Sebastian Rohjans, Andreas Rauh

benchmarks safety

arXiv Machine Learning

Jun 25

Onsager-Machlup Posterior Transport for Deep Gaussian Processes

arXiv:2605. 23434v2 Announce Type: replace Abstract: Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs).

By Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

diffusion benchmarks

arXiv Machine Learning

Jun 25

LLM Performance on a Real, Double-Marked GCSE Benchmark

arXiv:2606. 24973v1 Announce Type: cross Abstract: We introduce a dataset of 32,534 double-marked real student responses to GCSE mock exams (GCSEs are the UK's national exams, taken at age ~16), spanning 328 questions across five subjects and including handwritten work.

By Malachy Fox, Kavi Samra, Paul Jung

llms benchmarks

arXiv Machine Learning

Jun 25

Bridging Spherical Black-Box Optimizers

arXiv:2606. 25761v1 Announce Type: new Abstract: When gradient information is unavailable, black-box optimization (BBO) methods provide a practical alternative.

By Johannes Ackermann, Stefano Peluchetti

llms robotics multimodal benchmarks

arXiv Machine Learning

Jun 25

Frequency Domain Reservoir Computing

arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.

By Klaus Schertler, Xiomara Runge, Andrea Ceni, David Kappel, Claudio Gallicchio

llms rag benchmarks

arXiv Machine Learning

Jun 25

LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search

arXiv:2606. 25039v1 Announce Type: new Abstract: Recovering governing Ordinary Differential Equations (ODEs) from data is a central challenge in modeling dynamical systems across scientific domains.

By Nikhil Abhyankar, Sha Li, Sanchit Kabra, Naren Ramakrishnan, Yulia Gel, Chandan K. Reddy

llms benchmarks

arXiv Machine Learning

Jun 25

SDE-Driven Spatio-Temporal Hypergraph Neural Networks for Irregular Longitudinal fMRI Connectome Modeling in Alzheimer's Disease

arXiv:2603. 20452v2 Announce Type: replace Abstract: Longitudinal neuroimaging is essential for modeling disease progression in Alzheimer's disease (AD), yet irregular sampling and missing visits pose substantial challenges for learning reliable temporal representations.

By Ruiying Chen, Yutong Wang, Houliang Zhou, Wei Liang, Yong Chen, Lifang He

efficiency benchmarks

arXiv Machine Learning

Jun 25

RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments

arXiv:2606. 26094v1 Announce Type: new Abstract: For most of scientific history, researchers studying behavior could only infer hidden mechanisms from outward actions: an inverse problem that becomes more tractable when observation is augmented by targeted intervention.

By Babak Rahmani, Sebastian Dziadzio, Joschka Str\"uber, Sergio Hern\'andez-Guti\'errez, Matthias Bethge

llms agents benchmarks safety

arXiv Machine Learning

Jun 25

Latent Block-Diffusion Temporal Point Processes: A Semi-Autoregressive Framework for Asynchronous Event Sequence Generation

arXiv:2606. 24982v1 Announce Type: new Abstract: Modeling and sampling from the underlying distribution of asynchronous event sequences are crucial in various real-world applications, including social networks, medical diagnosis, and financial transactions.

By Shuai Zhang, Yancheng Chen, Chuan Zhou, Yang Liu, Xixun Lin, Xiangyu Zhao, Jun Zhu, Zhi-Ming Ma

diffusion benchmarks

arXiv Machine Learning

Jun 25

A Hybrid CNN-LSTM Intrusion Detection Framework for Cybersecurity in Smart Renewable Energy Grids

arXiv:2606. 25200v1 Announce Type: new Abstract: The accelerated digitalization of renewable energy smart grids through IoT sensors, AMI, and SCADA systems has significantly expanded the attack surface for sophisticated cyberattacks, FDI attacks that stealthily distort state estimation and DoS/DDoS attacks that flood communication channels.

By Sajib Debnath, Remon Das

efficiency benchmarks

Get the daily briefing