Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv Machine Learning
Oct 15

TopicMerge: A Divide and Conquer Approach to Topic Modeling

TopicMerge is a multi-stage pipeline that combines efficient embedding-based clustering with Large Language Model (LLM) guidance to produce high-quality topic models. The method first over-segments a corpus into fine-grained micro-clusters, then iteratively merges them using an LLM that evaluates candidate pairs based on local context. Experiments on Bills and Wiki datasets demonstrate state‑of‑the‑art clustering performance while cutting LLM calls by more than 92% compared to document‑level approaches.

By Matthew Raffel, Lizhong Chen, Sai Munikoti, Sameera Horawalavithana
arXiv Machine Learning
Oct 15

The Impact of Detector Architecture, Model Scale, and Scene Density on Multi-Pedestrian Tracking

The paper investigates how detector architecture, model scale, scene density, and inference regime influence multi‑pedestrian tracking performance. It compares YOLO11, RT‑DETR, and YOLO26 detectors with ByteTrack and BoT‑SORT trackers on MOT17, MOT20, and SOMPT22 datasets, finding RT‑DETR best on HOTA but at higher cost, while YOLO26‑L with NMS balances quality and speed. End‑to‑end inference for YOLO26 did not consistently improve tracking over NMS.

By Pedro Manoel Herminio Alves, Dalton C\'ezane Gomes Valadares, Cl\'audio de Souza Baptista
arXiv Machine Learning
Oct 15

State-Matched Sequence Conformal Prediction

State-Matched Sequence Conformal Prediction (SMSCP) is a new method for sequential conformal prediction that builds a one‑sided local‑linear state from pre‑target data and uses state similarity to select a matched reference set of past calibration scores. By applying an ordinary unweighted conformal order statistic to this matched set, SMSCP aims to better capture the target score quantile when comparable states recur at nonadjacent times. Experiments on solar irradiance and bike‑demand data show that SMSCP improves interval scores over recency, weighted, adaptive, and HopCPT methods, and achieves coverage closer to the nominal 0.90 level than ResCP on most benchmarks.

By Ruihang Han, Li-Hsiang Lin
arXiv Machine Learning
Oct 15

INTACT: Inducing Latent Reasoning on a Frozen Backbone

INTACT introduces a small external encoder that induces latent reasoning in a frozen backbone model by mapping hidden states to compact continuous representations, called soft-token blocks. The encoder is trained via distillation to mimic written reasoning steps while preserving the backbone’s behavior, and then fine‑tuned for answer correctness. Experiments on four frozen backbones (1B–8B) show that INTACT improves accuracy on GSM8K and other out‑of‑domain benchmarks, reduces latency by up to 19%, and outperforms existing latent‑reasoning baselines at 3B and 8B scales.

By Kutay Tire, Atula Tejaswi, Haris Vikalo, Sujay Sanghavi
arXiv Machine Learning
Oct 15

Who's Behind the Text? Robust Model Authorship Attribution

The paper introduces ATLAS, a likelihood‑based method for model authorship attribution (MAA) that learns how each source family generates text and attributes new text by comparing token‑level likelihoods. It also presents MAABench, a benchmark for evaluating MAA under out‑of‑distribution scenarios such as unseen domains and future model versions. Experiments show ATLAS outperforms existing baselines, achieving 74.27% accuracy on the hardest MAABench setting and correctly identifying teacher models in distillation experiments.

By Minseok Ko, Hyunseok Lee, Woomin Song, Beomjun Kim, Jinwoo Shin, Kunwoong Kim
arXiv Machine Learning
Oct 15

StateCraft: Augmenting Video World Models with Abstract-State Reasoning

StateCraft introduces a new architecture that separates a persistent, updatable abstract state from the visual state in video world models. By pairing a language model with a video diffusion model, it tracks entity changes over time and conditions visual generation on this abstract state. The approach is evaluated on generative interfaces and a new benchmark, showing improved generation quality, memory usage, and controllability, while enabling direct editing of unseen states and extending to spatial 3D environments.

By Lorenzo Venturoli, Nedko Savov, Xi Wang, Marc Pollefeys, Luc Van Gool
arXiv Machine Learning
Oct 15

Text-Informed Training Data Filtering for Enhanced Virtual Flow Meter Performance

The paper introduces a text-informed training data filtering approach that uses large language models to combine unstructured operational text with tabular sensor data for anomaly detection and dataset masking. Applied to a virtual flow meter for oil and gas, the multimodal method reduces mean absolute percentage error from 7.1% to 5.4% compared to sensor-only filtering. Ablation studies confirm that operator comments provide additional filtering signals beyond sensor data alone.

By Kristoffer Hjelmen Nesland (University of Stavanger, Stavanger, Norway, Solution Seeker AS, Oslo, Norway), Bjarne Grimstad (Solution Seeker AS, Oslo, Norway), Antorweep Chakravorty (University of Stavanger, Stavanger, Norway), Dan Sui (University of Stavanger, Stavanger, Norway)
arXiv Machine Learning
Oct 15

SAM3RL: Memory Control via Reinforcement Learning for Visual Object Tracking

SAM3RL introduces a reinforcement‑learning based memory controller for the Segment Anything Model 3 (SAM 3) visual object tracker, deciding which past frames to store in a memory bank to improve tracking performance. This data‑driven approach replaces hand‑crafted memory update rules and achieves state‑of‑the‑art results on 10 of 12 tracking and video object segmentation benchmarks, with no runtime overhead. The same method applied to SAM 2 (SAM2RL) also outperforms hand‑crafted rules on 9 of 12 benchmarks, demonstrating its generality across model generations.

By Tom\'a\v{s} \v{C}\'i\v{z}ek, Matej Straka, Illia Volkov, Mat\v{e}j Kripner, Ervin Maci\'c, Jiri Matas, Klara Janouskova, Martin Schmid
arXiv Machine Learning
Oct 15

Contrastive uncertainty learning for robust multi-structure segmentation in lower-pelvic MR

The paper introduces Contrastive Uncertainty Learning (CUL), a supervised framework that leverages voxel‑wise predictive uncertainty to guide representation learning for multi‑structure segmentation in lower‑pelvic MRI. Using Monte Carlo Dropout, CUL identifies uncertain voxels and shapes the embedding space so that low‑uncertainty, correctly predicted features become stable while high‑uncertainty or misclassified features are repelled, preventing feature collapse at challenging boundaries. Evaluated on 589 prostate MRI scans with a ViT‑based UperNet decoder, CUL achieves comparable Dice scores to a baseline while significantly reducing surface distance for most structures, and ablation studies confirm the necessity of both uncertainty‑guided sampling and the contrastive objective.

By Bilal Sidiqi, Yipei Wang, Wen Yan, Shaheer U. Saeed, Daniel C. Alexander, Yipeng Hu
arXiv Machine Learning
Oct 15

VASPO: Self-Distillation with Privileged Value Estimation

VASPO introduces a new integration of self-distillation and PPO by leveraging privileged value estimation to guide language model training. The method uses a self-teacher and a critic that both evaluate the same privileged information on student-generated prefixes, allowing the critic to compute token-level PPO advantages from predicted continuation returns while the teacher modulates positive advantages via bounded probability ratios. Experiments on six mathematical reasoning benchmarks show that VASPO improves mean accuracy over standard PPO by 2.59 and 1.81 percentage points on Qwen3-4B and OLMo3-7B, respectively, and achieves the highest mean accuracy among evaluated methods on Knights and Knaves.

By Fei Wu, Zhenrong Zhang, Jianshu Zhang, Jun Du
arXiv Machine Learning
Oct 15

Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter

Sig-DEG for Distillation proposes a Signature-based Differential Equation Generator to distill pre-trained diffusion models. By combining partial signatures of Brownian motion with a recurrent structure, it approximates stochastic differential equation solutions and formulates distillation as supervised learning on coarse time grids. The method enables fast inference, achieving up to several hundred‑fold speedups while maintaining competitive sample quality in image and time‑series generation, and extends to applications like Schrödinger bridges.

By Wen Ge, Lei Jiang, Lingyi Yang, Leonardo Roveri, Mingxuan Yi, Po-Yu Chen, Niels Cariou-Kotlarek, Francois Buet-Golfouse, Gaurav Mittal, Hao Ni
arXiv Machine Learning
Oct 15

A Byzantine-Robust Federated Learning with Nesterov Momentum

The paper introduces Byrd-NAFL, a Byzantine‑robust federated learning algorithm that incorporates Nesterov momentum at the server while keeping workers stateless. It applies any soft γ‑Byzantine resilient aggregation rule and provides finite‑time convergence guarantees for smooth non‑convex losses across all momentum factors β∈[0,1). Experiments on MNIST, CIFAR‑10, and Sentiment140 demonstrate faster convergence and higher accuracy than existing benchmarks under various attacks.

By Lihan Xu, Xiaoyi Fan, Gang Wang, Runhao Zeng, Xiping Hu, Yanjie Dong
arXiv Machine Learning
Oct 15

Constitutive parameterized deep energy method for solid mechanics problems with random material parameters

The paper introduces the Constitutive Parameterized Deep Energy Method (CPDEM), a physics‑driven neural network that embeds stochastic material parameters directly into the strain energy density functional. By training via expected energy minimization over the parameter domain, CPDEM learns a continuous solution manifold, enabling zero‑shot, real‑time inference of displacement fields for any material parameters without retraining or dataset generation. The method is validated on linear elasticity, finite‑strain hyperelasticity, and highly nonlinear contact mechanics, marking the first purely physics‑driven deep learning approach that efficiently handles continuous multi‑parameter variations in solid mechanics.

By Zhangyong Liang, Jihong Zhu, Huanhuan Gao
arXiv Machine Learning
Oct 15

Medical Imaging Classification with Cold-Atom Reservoir Computing using Auto-Encoders and Surrogate-Driven Training

The paper presents a hybrid quantum‑classical pipeline that uses neutral‑atom reservoir computing for medical image classification, specifically binary polyp detection. It incorporates a guided auto‑encoder to reduce dimensionality and a differentiable surrogate model to bypass the gradient barrier of quantum measurements, enabling end‑to‑end backpropagation. The resulting compact latent representations are encoded as pulse detuning parameters in a Rydberg Hamiltonian, producing quantum embeddings that feed a linear classifier, and simulations show improved performance over traditional PCA or unguided auto‑encoder methods.

By Nuno Batista, Ana Morgado, Oscar Ferraz, Sagar Silva Pratapsi, Jorge Lobo, Gabriel Falcao
arXiv Machine Learning
Oct 15

TIDES: Implicit Time-Awareness in Selective State Space Models

TIDES is a new selective state space model that keeps the learned step size equal to the physical time gap, enabling it to handle irregular timestamps while maintaining per‑token expressivity. By moving input dependence from the step size to the diagonal state matrix, TIDES combines the strengths of selective SSMs like Mamba and continuous‑time SSMs like S5. Experiments on a novel Fading Flash benchmark and large‑scale datasets show that TIDES outperforms or matches state‑of‑the‑art models on irregular time‑series tasks.

By Taylan Soydan, Miguel A. Bessa, Dirk Mohr, Rui Barreira