← Back to all news

Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Machine Learning
Oct 8

Few-Shot Learning for Personalised Automated Pain Assessment

The paper investigates using Few-Shot Learning, a meta‑learning technique, to create personalised classifiers for automated pain assessment. By treating the shift from population‑level to subject‑level evaluation as a task‑domain shift, the authors evaluate their approach on three datasets—BioVid, SenseEmotion, and PainMonit—achieving high accuracies in both binary and multi‑class settings, with further improvements when using k‑shot conditioning. Ablation studies confirm that support‑conditioned few‑shot adaptation enhances performance across subjects.

By Heinke Hihn, Ibrahim Eisawy, Patrick Thiam, Hans A. Kestler, Friedhelm Schwenker
benchmarks
More like this →
arXiv Machine Learning
Oct 8

RSIGym: A Flexible Environment for Recursive Self-Improvement

RSIGym is a new agent-native research environment built on Everything as a Service (EaaS) that exposes training, inference, rollout, evaluation, and sandbox execution through reusable services. It supports shared budget and permission controls for Data, Harness, and Joint improvement tracks, allowing agents to investigate individual interventions and jointly optimize data, training settings, and execution harnesses within the same environment. Using RSIGym, the Opus 5 model achieved the highest RSI-Index of 0.4809 across five benchmarks, improving performance on software engineering, terminal interaction, mathematics, scientific reasoning, and skill-based tasks.

By Fanqing Meng, Lingxiao Du, Haocheng Lu, Qiguang Chen, Ziqi Zhao, Zijian Wu, Jiayuan Zhuo, Mengkang Hu, Michael Qizhe Shieh
agentsbenchmarks
More like this →
arXiv Computer Vision
Oct 8

VolCo: Volumetric Contact for High-Fidelity Human Grasp Generation

The paper introduces VolCo, a volumetric contact representation that extends surface points into 3D grids to model hand-object interactions more accurately. Using a 3D variational autoencoder for local contact details and a prior-guided diffusion model for global hand geometry, the VolCoDiff framework achieves tighter grasps with reduced penetration on benchmark datasets. The authors provide code for reproducibility.

By Zhuo Chen, Yihua Cheng, Ale\v{s} Leonardis, Hyung Jin Chang
diffusionbenchmarks
More like this →
arXiv Machine Learning
Oct 8

VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation

VM-ARRAYDPS introduces virtual microphones with higher signal‑to‑noise ratios to augment a limited physical microphone array, providing additional multi‑channel consistency constraints for blind speech separation. By leveraging a pretrained speech diffusion model and posterior sampling, the method improves upon the existing ArrayDPS approach. Experiments on 2‑speaker and 3‑speaker datasets show significant performance gains, and ablation studies reveal the impact of virtual microphone count and consistency weighting.

By Jingqi Sun, Haozhan Tang, Shulin He, Zhong-Qiu Wang
diffusionbenchmarks
More like this →
arXiv Machine Learning
Oct 8

Correspondences as Decisions: JevNexus for Decision-Centric Schema Matching

The paper introduces JevNexus, a decision‑centric schema‑matching approach that uses typed pairwise decisions combined with schema and instance evidence, and only triggers listwise refinement when evidence conflicts and the fused margin is small. In experiments on 561 cases across six benchmark families, JevNexus achieves a dataset‑macro MRR of 0.930 and Hits@1 of 0.909, slightly outperforming Magneto while dramatically reducing mean latency from 123.452 to 15.929 seconds. The method activates listwise refinement for only 5.665% of source columns, avoiding the performance degradation seen with unconditional refinement.

By Runze Li, Hanchen Wang, Ying Zhang, Wenjie Zhang
llmsbenchmarks
More like this →
arXiv Machine Learning
Oct 8

Latent Performance Profiling of Large Language Models

arXiv:2605.30018v3 Announce Type: replace-cross Abstract: Large language models (LLMs) frequently achieve impressive scores on standardized benchmarks, yet accuracy alone offers a limited view of the...

By Tanmoy Chakraborty, Ayan Sengupta, Suparna Bhattacharya, Partha Pratim Chakrabarti, Amlan Chakrabarti, Supratik Chakraborty, Partha Pratim Das, Lipika Dey, Richa Singh, Mayank Vatsa
llmsbenchmarkssafety
More like this →
arXiv Machine Learning
Oct 8

A Unified Information-Theoretic Approach to Constrained Multi-Fidelity Multi-Objective Bayesian Optimization

The paper introduces a unified information‑theoretic framework for Bayesian optimization that simultaneously handles multiple objectives, constraints, and fidelity levels. It defines query utility as the information gain about the highest‑fidelity feasible Pareto frontier and derives a tractable variational lower bound using truncated approximations. The resulting cost‑aware acquisition function leverages multi‑fidelity surrogate models, eliminating the need for separate heuristics, and is validated on synthetic, benchmark, and real‑world experiments.

By Rikuto Matsumoto, Masanori Ishikura, Masayuki Karasuyama
benchmarks
More like this →