Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Machine Learning
Sep 29

From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning

The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.

By Yichen Lin, Xuyuan Xiong, Xue Wang, Xiangfu Meng, Mike Mingcheng Wei, Tao Yao
arXiv Machine Learning
Sep 29

Fake News Theories: Harnessing Disciplinary Insights for Computational Modeling, Detection, and Explanation

The paper presents a theory-informed computational framework that converts cross-disciplinary theories of fake news into measurable features for automated detection and explanation. By reviewing theories from social sciences, psychology, economics, and more, the authors establish a broad theoretical foundation for computational modeling. Experiments on benchmark datasets demonstrate that theory-derived features are predictive, provide interpretable diagnostic signals, and that multi-feature models generally outperform individual features, though gains are modest.

By Zhaoyang Cao, Miriam Metzger, Reza Zafarani
arXiv Machine Learning
Sep 29

Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis

Reliability-aware Cross-sample Enhancement (RCE) is a framework for multimodal sentiment analysis that tackles noise and missing modalities by first applying an adaptive variational information bottleneck to compress unreliable modality information. It then retrieves high‑confidence, semantically consistent neighbors from a large candidate pool to enrich current representations, and finally fuses cross‑modal interactions through a multilevel reliability‑aware mechanism. Experiments show RCE consistently outperforms state‑of‑the‑art methods in full, noisy, and missing‑modality scenarios.

By Menghua Jiang, Haokai Gao, Xiangui Kang, Haifeng Hu, Sijie Mai
arXiv Machine Learning
Sep 29

Benchmarking the Connectomes of Caenorhabditis elegans within the Reservoir Computing Framework

The paper investigates the connectomes of *Caenorhabditis elegans* by implementing them as echo state networks within a reservoir computing framework. Using connectomes derived at different ages and through three distinct measurement methods, the authors benchmark performance on neuro-inspired tasks, comparing biological wiring to randomized null models. Results indicate that biological wiring and bio-informed input/output configurations do not consistently outperform random models, and performance varies significantly with reservoir configuration and connectome derivation.

By Felix S. Reimers, Ola Huse Ramstad, Aliaksandr Hubin, Stefano Nichele
arXiv Machine Learning
Sep 29

Probabilistic Robustness-driven Universal Adversarial Perturbations with Explainability against Deep Reinforcement Learning-based Intrusion Detection System

The paper introduces a new method for generating universal adversarial perturbations (UAPs) against deep reinforcement learning (DRL)-based intrusion detection systems (IDS). It leverages Probabilistic Robustness (PR) as a post‑hoc metric to guide UAP creation, integrating PR directly into the optimization objective. The authors further develop PX‑UAP, which incorporates explainable AI (XAI) to shape perturbations within realistic domain constraints, and provide a theoretical analysis of its design. Experiments show PX‑UAP outperforms existing UAP techniques in attack effectiveness.

By Hongsen Zhang, Lu Zhang, Mingjing Xu, Yi Zhang, Gregory Epiphaniou, Carsten Maple
arXiv Machine Learning
Sep 29

Mechanism-Aware Ensemble Conditioning for Data-Limited Emulation of Extreme Events

The paper introduces a mechanism‑aware conditioning framework that uses a nudged coarse ensemble to capture local instability geometry in chaotic systems. By injecting ensemble covariance statistics via a small FiLM module, the authors enhance rare‑event emulation in both a low‑dimensional chaotic benchmark and a quasi‑geostrophic flow model, achieving significant improvements in exceedance‑frequency and tail‑density errors with limited data. The approach demonstrates that local instability information can be leveraged as a practical conditioning signal for data‑efficient emulation of extreme events.

By Isabella S. Thiel, Juan Bello-Rivas, Yannis G. Kevrekidis, Themistoklis P. Sapsis
arXiv Machine Learning
Sep 29

Learning Provable Neural Network Observer for Uncertain Dynamical Systems

The paper introduces a two‑stage training framework for neural network observers that guarantees Lyapunov stability for uncertain dynamical systems. First, a point‑guided Lyapunov pre‑training phase quickly achieves high estimation accuracy and local stability over sampled states. Second, an LMI fine‑tuning phase efficiently enforces a strict global Lyapunov stability certificate, yielding provably stable observers that train faster than direct LMI methods and generalize robustly across diverse systems.

By Zhangyi Wang, Jiaxu Liu, Chen Song, Chao Xu, Shengze Cai
arXiv Machine Learning
Sep 29

Robust Graph Clustering Network for Multiple Missing Data

The paper introduces the Robust Graph Clustering Network for Multiple Missing Data (RGCN), a method designed to cluster graphs with simultaneous missing node attributes and structural links. RGCN employs a view‑decoupled dual‑branch imputation to reduce cross‑view interference, a multi‑hyperspherical mixture prior to improve cluster compactness and separability on a directional latent manifold, and a boundary‑aware contrastive enhancement objective to counteract cluster blurring caused by imputation bias. Experiments on real‑world datasets show that RGCN consistently outperforms state‑of‑the‑art baselines across various missing data patterns.

By Keyuan Qiu, Renda Han, Zhen Tang, Qiang He, Xingwei Wang, Wenxin Zhang, Guangzhen Yao, Junxin Chen, Qingjian Ni
arXiv Machine Learning
Sep 29

Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra

The paper presents a self‑supervised approach to learning representations of auroral emission spectra using a 1D Vision Transformer trained with a masked autoencoder on 223,000 unlabelled spectra. The pretrained model recovers key emission‑line intensity ratios with high accuracy (R² = 0.91) and, with a single linear probe, matches expert‑designed feature classifiers. When fine‑tuned, it surpasses the previous supervised auroral classifier (macro‑AP 88.5 vs. 77.8) and achieves a 0.870 mAP, outperforming a model trained from scratch by 0.159 using only 10 % of the labels, while attribution reveals reliance on N₂⁺ bands.

By Matthieu Le Lain, Ga\"el Cessateur, S\'ebastien Lef\`evre
arXiv Machine Learning
Sep 29

I Act Therefore I Am: When Is JEPA's Action-Conditioning Enough to Learn Causal Mechanisms?

The paper studies when joint-embedding predictive architectures (JEPAs) can recover underlying causal states from high‑dimensional observations. It introduces a latent variable model where observations arise from causal states with action‑conditioned dynamics, and proposes an information‑theoretic objective that maximizes conditional likelihood while preserving state entropy. The authors prove identifiability conditions—particularly sufficient action‑induced variation—and instantiate the objective as an action‑modulated Gaussian additive‑noise model (A‑JEPA), demonstrating theoretical and empirical success in synthetic and visual benchmarks.

By Yuhang Liu, Zhuo Huang, Javen Qinfeng Shi
arXiv Machine Learning
Sep 29

Bridging Body and Brain: Gene-Driven Morphology--Control Co-Design

The paper introduces Morphogene, a compact latent blueprint that links an agent’s body and control policy, enabling coordinated changes at the limb level through AdaConcat. Building on this, GeCode treats co-design as exploration within Morphogene space, using local refinement and global exploration to efficiently navigate design regions. Experiments on various 2D and 3D tasks show that GeCode outperforms state‑of‑the‑art methods, achieving faster convergence and higher performance.

By Fu Feng, Ruixiao Shi, Yucheng Xie, Jing Wang, Xin Geng
arXiv Machine Learning
Sep 29

More Sensors Only One Field: Rethinking Continual Spatio-Temporal Forecasting

The paper introduces STFO (Spatio-Temporal Field Operator), a method for continual spatio‑temporal forecasting that decouples forecasting representations from specific sensor layouts. By normalizing coordinate‑based aggregation onto a fixed latent grid and using a spectral descriptor to adapt to process drift, STFO can reuse learned spatial maps across varying sensor configurations. Experiments on PEMS‑Stream, CA‑Stream, and AIR‑Stream show that STFO‑Large improves average MAE by 8.4% over DOL on PEMS‑Stream and 4.7% on CA‑Stream.

By Lewei Xie, Haoyu Zhang, Jiajun Zhou, Yulong Chen, Guanxing Chen, Yu-An Huang, Hau-San Wong, Yifan Zhang, Zhi-An Huang
arXiv Machine Learning
Sep 29

Learning Chance-Constrained MDPs with Bellman Distributional Certificates

The paper introduces a new approach to learning chance-constrained Markov decision processes (CCMDPs) using a Bellman distributional certificate. It provides both model-based and model-free algorithms with theoretical guarantees, including matching upper and lower bounds for tabular discounted CCMDPs with bounded successor support. Numerical experiments on synthetic CCMDPs and an IEEE 14-bus energy storage benchmark demonstrate the safety and effectiveness of the proposed methods.

By Chenbei Lu, Hongyu Yi
arXiv Machine Learning
Sep 29

Seeing Speech: Learning Visible Articulatory Dynamics for Speech-Driven 3D Facial Animation

The paper introduces a new framework for speech‑driven 3D facial animation that explicitly models visible articulatory dynamics. It uses a Speech‑Articulatory Memory (SAM) to link speech to three directional articulatory motions—spreading, opening, and protrusion—under phonetic context, and a Topology‑aware Articulatory Composition (TAC) to integrate these motions into surface‑consistent facial motion. Experiments on VOCASET and TFHP demonstrate state‑of‑the‑art reconstruction quality and improved lip articulation metrics, with a user study confirming better lip sync and realism.

By Hyung Kyu Kim, Byungchan Hwang, Hak Gu Kim
arXiv Machine Learning
Sep 29

Differentially-Private Decision Trees and Provable Robustness to Data Poisoning

The paper introduces PrivaTree, a differentially‑private decision tree algorithm that uses private histograms to select splits while preserving a small privacy budget. PrivaTree supports mixed numerical and categorical data without leaking information about numerical features and achieves a superior privacy‑utility trade‑off compared to existing methods. Additionally, the authors provide theoretical bounds on the expected accuracy and success rates of backdoor attacks, showing that PrivaTree-trained trees are more robust against data poisoning than standard decision trees.

By Dani\"el Vos, Jelle Vos, Tianyu Li, Zekeriya Erkin, Sicco Verwer
arXiv Machine Learning
Sep 29

FLAME: Flow Enhanced Legendre Memory Models for General Time Series Forecasting

FLAME is a lightweight Time Series Foundation Model that uses Legendre Memory variants (LegT and LegS) in its encoding and decoding stages to capture inductive biases and perform efficient long‑range forecasting. It incorporates a normalizing‑flow forecasting head to generate complex probabilistic distributions over future horizons. Experiments on TSFM‑Bench, ProbTS, and TFB show FLAME performs strongly as an out‑of‑the‑box tool for decision intelligence.

By Xingjian Wu, Zhengyu Li, Hanyin Cheng, Xiangfei Qiu, Jilin Hu, Chenjuan Guo, Bin Yang
arXiv Machine Learning
Sep 29

Distribution-Conditioned Transport

The paper introduces Distribution‑Conditioned Transport (DCT), a framework that learns transport maps conditioned on embeddings of source and target distributions, allowing generalization to unseen distribution pairs. DCT supports semi‑supervised learning for distributional forecasting by leveraging distributions observed at only one condition. It is agnostic to the transport mechanism and is demonstrated on synthetic benchmarks and four biological applications, including batch effect transfer in single‑cell genomics and modeling T‑cell receptor sequence evolution.

By Nic Fishman, Gokul Gowri, Paolo L. B. Fischer, Marinka Zitnik, Omar Abudayyeh, Jonathan Gootenberg
arXiv Machine Learning
Sep 29

Low-Cost Black-Box Detection of LLM Hallucinations via Dynamical System Prediction

The paper introduces a low-cost method for detecting hallucinations in large language models by treating the model as a black-box dynamical system. It projects responses into a high-dimensional manifold, models the latent state-space dynamics with Koopman operator theory, and uses differential residual scores from transition operators to distinguish factual from hallucinated outputs. The approach requires only a single-sample pass and shows state-of-the-art performance across three benchmarks with reduced resource overhead.

By Dan Wilson, Mohamed Akrout
arXiv Machine Learning
Sep 29

Tabular Imbalanced Learning: A Survey, Benchmark, and Practical Guide

The paper surveys tabular imbalanced learning and introduces TILBench, a benchmark evaluating over 40 methods on 57 datasets. It presents a unified taxonomy of approaches and shows that no single method dominates across all settings, with performance depending on dataset regimes and computational constraints. Practical recommendations for method selection and future research directions are provided.

By Ruizhe Liu, Jiaqi Luo