Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Computer Vision
Sep 22

Residual Primitive Fitting of 3D Shapes with SuperFrusta

arXiv:2512.09201v2 Announce Type: replace-cross Abstract: We introduce a framework for converting 3D shapes into compact and editable assemblies of analytic primitives, directly addressing the persis...

By Aditya Ganeshan, Matheus Gadelha, Thibault Groueix, Zhiqin Chen, Siddhartha Chaudhuri, Vladimir G. Kim, Wang Yifan, Daniel Ritchie
arXiv Machine Learning
Sep 22

NAVIR: Neuromorphic Audio-Visual Speech Recognition for Robust Human-Robot Interaction on Edge Hardware

NAVIR is an end‑to‑end audio‑visual speech recognition system designed for the BrainChip Akida neuromorphic processor, which only supports sequential 2‑D convolutions. The architecture separates spatial and temporal encoding into three AkidaNet modules—per‑frame visual, temporal video, and spectrogram audio encoders—fused by a lightweight predictor and decoded with constrained beam search. Trained with CTC on noise‑augmented audio and fine‑tuned via quantization‑aware training, the quantized model achieves 14.0% WER on GRID’s unseen‑speaker split and 3.3% on overlapped‑speaker split, outperforming audio‑only baselines, and delivers 98.6% command accuracy at 1.5% WER on an industrial‑command corpus, while offering a 13‑fold energy advantage over conventional ANNs and roughly 5‑fold lower energy per inference than a Raspberry Pi CPU.

By Leonidas Delimpasis, Panagiota Moraiti, Antonis Porichis, Panos Chatzakos, Michail Karamousadakis
arXiv Machine Learning
Sep 22

Stochastic Reconfiguration as Statistical Filtering for Overparameterized Neural Quantum States

The paper investigates how stochastic reconfiguration (SR), the standard optimizer for neural quantum states (NQS), functions as a statistical filter in overparameterized regimes where parameters outnumber Monte Carlo samples. By interpreting SR as ridge regression on tangent features, the authors show that the diagonal shift balances useful update directions against variance from fitting finite-sample residuals, leading to a U-shaped validation risk curve. They introduce multi-shift SR (MS‑SR), which averages ridge solutions at data‑adaptive shifts, and demonstrate that it reduces validation risk and update variance compared to fixed‑shift SR in both small and large system experiments.

By Tak Hur
arXiv Machine Learning
Sep 22

Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea

The study benchmarks Vision Transformers (ViTs) against convolutional neural networks (CNNs) for fine‑grained orchid genus identification in New Guinea’s species‑rich, data‑poor flora. Using a two‑stage system that first predicts genus and then retrieves similar species images, the authors fine‑tuned four pretrained backbones on 16,701 photographs from 120 genera and 1,350 species. The self‑supervised ViT DINOv2 achieved the highest genus accuracy (macro top‑1 66.9 %) and outperformed both CNNs and a domain‑matched pretrained ViT, demonstrating strong species retrieval and open‑set detection capabilities.

By Reza Saputra, Diah Harnoni Apriyanti, Andr\'e Schuiteman, Kurt Metzger, Ashley Field, Katharina Nargar, William Edwards