Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

19,767 stories · RSS feed

arXiv Machine Learning

Jun 25

Multi-Stream Temporal Fusion for Financial Fraud Detection

arXiv:2606. 25007v1 Announce Type: new Abstract: Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams -- transactions, login sessions, risk signals -- that individually appear benign but collectively reveal fraudulent patterns.

By Mohammadamin Dashti Moghaddam, Nick Sciarrilli

llms benchmarks safety

arXiv Machine Learning

Jun 25

Swarm-Inspired Generation of Collective Behaviors in Graph Dynamical Systems

arXiv:2606. 24958v1 Announce Type: new Abstract: Collective behavior arises when locally interacting units produce coordinated global organization, from synchronization in dynamical systems to task-relevant information flow on graphs.

By Ji Chen, Song Chen, Chengzhang Gong, Li Fan, Chao Xu

agents robotics benchmarks

arXiv Machine Learning

Jun 25

Hierarchical Graph Learning for Calendar Spread Strategies in Commodity Futures Markets

arXiv:2606. 25811v1 Announce Type: cross Abstract: Commodity futures can be represented hierarchically, with underlying assets at the upper level and individual futures contracts at the lower level.

By Yoonsik Hong, Diego Klabjan

benchmarks

arXiv Machine Learning

Jun 25

LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learning

arXiv:2606. 24901v1 Announce Type: new Abstract: Continual learning capability is critical for Industrial LLMs, as deployed models must be continuously updated to meet evolving requirements and environments, rather than repeatedly retrained from scratch.

By Hao Jiang, Enneng Yang, Guojie Zhu, Yibin Chen, Yunkun Xu, Zifu Kou, Jiayi Li, Chong Chen, Zhao Cao, Li Shen

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

RN-D: Discretized Categorical Actors for On-Policy Reinforcement Learning

arXiv:2601. 23075v2 Announce Type: replace Abstract: On-policy Reinforcement Learning (RL) remains a dominant paradigm for continuous control, yet standard implementations rely on Gaussian actors and relatively shallow MLP policies, often leading to brittle optimization when gradients are noisy, and policy updates must be conservative.

By Yuexin Bian, Jie Feng, Tao Wang, Yijiang Li, Sicun Gao, Yuanyuan Shi

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion

arXiv:2606. 25097v1 Announce Type: new Abstract: Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs?

By Sahil Kadadekar

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 25

Improved Large Language Diffusion Models

arXiv:2606. 25331v1 Announce Type: cross Abstract: Modern large language models are predominantly trained with autoregressive factorization and causal attention.

By Shen Nie, Qiyang Min, Shaoxuan Xu, Zihao Huang, Yuxuan Song, Yong Shan, Yankai Lin, Wayne Xin Zhao, Chongxuan Li, Ji-Rong Wen

llms diffusion fine-tuning benchmarks

arXiv Machine Learning

Jun 25

ATMA: Length-Invariant Language Modeling via Polar Attention and Gated-Delta Compression Memory

arXiv:2606. 25156v1 Announce Type: new Abstract: Modern large language models based on softmax scaled-dot-product attention are constrained by their training sequence length: as the key-value sequence grows, softmax probability mass can dilute across a wider distribution, inducing activation shift and long-context performance collapse.

By Habibullah Akbar

llms benchmarks

arXiv Machine Learning

Jun 25

From Forecasting Leaderboards to Deployment Decisions: A Fail-Closed Certification Protocol

arXiv:2606. 24996v1 Announce Type: new Abstract: Forecasting leaderboards rank models by predictive quality, but their winners are often read as deployment-ready top-1 advice.

By Geumyoung Kim

benchmarks

arXiv Machine Learning

Jun 25

Model-agnostic Mitigation Strategies of Data Imbalance for Regression

arXiv:2506. 01486v2 Announce Type: replace Abstract: Data imbalance persists as a pervasive challenge in regression tasks, introducing bias in model performance and undermining predictive reliability.

By Jelke Wibbeke, Sebastian Rohjans, Andreas Rauh

benchmarks safety

arXiv Machine Learning

Jun 25

Pulmonary Embolism Risk Stratification from CTPA and Medical Records: Vascular Graphs Are Not All You Need

arXiv:2606. 25956v1 Announce Type: cross Abstract: Risk stratification for pulmonary embolism (PE) is critical for clinical decision-making.

By Nathan Painchaud, Tristan Hab\'emont, Morgane des Ligneris, Allan Serva, Pierre Croisille, Laurent Bertoletti, Thomas Lampert, Johannes F. Lutzeyer, Odyss\'ee Merveille

benchmarks

arXiv Machine Learning

Jun 25

Supervised Reinforcement Learning for the Coordination of Distributed Energy Resources

arXiv:2606. 24947v1 Announce Type: new Abstract: The increasing integration of distributed energy resources (DERs) is crucial for power system decarbonization, yet unlocking DERs' flexibility is challenged by their inherent uncertainties and modelling complexity.

By Haoyuan Deng, Yihong Zhou, Thomas Morstyn, Yi Wang

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 25

PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution

arXiv:2507. 01695v3 Announce Type: replace Abstract: Deep neural networks (DNNs) are widely used for their ability to model complex patterns across domains such as computer vision, speech recognition, and robotics.

By Omkar Shende, Gayathri Ananthanarayanan, Marcello Traiola

computer-vision robotics multimodal benchmarks

arXiv Machine Learning

Jun 25

How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly Dissolves

arXiv:2606. 25092v1 Announce Type: new Abstract: Sparse Mixture-of-Experts (MoE) models route each token to a few of many experts, inviting the hypothesis that experts form functional modules tied to capabilities or languages.

By Tony Salomone, Deep Gandhi, Ali Asaria

efficiency benchmarks

arXiv Machine Learning

Jun 25

Forget to Improve: On-Device LLM-Agent Continual Learning via Budget-Curated Memory

arXiv:2606. 25115v1 Announce Type: new Abstract: On-device language-model agents improve by accumulating experience in retrieved memory rather than by updating weights.

By Beining Wu, Zihao Ding, Jun Huang, Yanxiao Zhao

llms agents robotics benchmarks

arXiv Machine Learning

Jun 25

Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning

arXiv:2510. 04773v2 Announce Type: replace Abstract: As Large Language Models (LLMs) demonstrate remarkable capabilities learned from vast corpora, concerns regarding data privacy and safety are receiving increasing attention.

By Kai Qin, Jiaqi Wu, Jianxiang He, Haoyuan Sun, Yifei Zhao, Xu Wang, Bin Liang, Yongzhe Chang, Cheng Li, Tiantian Zhang, Houde Liu

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 25

Bellman-sufficient Information Complexity

arXiv:2606. 11171v4 Announce Type: replace Abstract: We develop Bellman-sufficient information complexity, a formal representation-level framework for sequential decision making.

By Yunbei Xu

benchmarks

arXiv Machine Learning

Jun 25

MINIF2F-DAFNY: LLM-Guided Mathematical Theorem Proving via Auto-Active Verification

arXiv:2512. 10187v3 Announce Type: replace Abstract: LLMs excel at reasoning, but validating their steps remains challenging.

By Mantas Baksys, Stefan Zetzsche, Olivier Bouissou, Sean B. Holden

llms benchmarks

arXiv Machine Learning

Jun 25

Polaris: A G\"odel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair

arXiv:2603. 23129v4 Announce Type: replace Abstract: G\"odel agent realize recursive self-improvement: an agent inspects its own policy and traces and then modifies that policy in a tested loop.

By Aditya Kakade, Vivek Srivastava, Shirish Karande

llms agents benchmarks

arXiv Machine Learning

Jun 25

Efficient Adaptive Data Acquisition via Pretrained Belief Representations

arXiv:2606. 25197v1 Announce Type: new Abstract: Learning effective policies for adaptive data acquisition remains challenging: posterior-based methods rely on surrogate models and posterior approximations that can be misspecified or biased, while direct policy-learning methods map from historical observations and fail to exploit available model representations, making learning harder.

By Daolang Huang, Zhuoyue Huang, Conor Hassan, Luigi Acerbi, Samuel Kaski, Tom Rainforth

benchmarks

Get the daily briefing