Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Machine Learning
Oct 8

Efficient Patch-Based Anomaly Detection Fused with Diffusion Driven Generative Modeling for Semiconductor Wafer Bin Map Open Set Anomaly Detection

The paper introduces a hybrid one‑class anomaly detector for semiconductor wafer bin maps that combines a patch‑based student‑teacher model (EfficientAD) with a denoising diffusion probabilistic model (DDPM). Trained on only 700 normal wafers, the fused system achieves an AUROC of 0.9985 on 18,658 held‑out wafers, reducing misclassifications from 852 (DDPM) and 1,412 (EfficientAD) to 618, with statistically significant improvements. Analysis reveals that the two modules make weakly overlapping errors, and that fixed‑weight fusion captures only 40‑70 % of the potential correction, highlighting the need for sample‑adaptive fusion and imbalance‑aware evaluation.

By Limon Bin Hossain, Md Sadib Rahman Ananta
arXiv Machine Learning
Oct 8

CAFE+FNO: Fourier Kernel Generation via Multiplicative Feature Composition

The paper introduces CAFE+FNO, a Fourier Neural Operator variant that enhances kernel generation by combining Fourier–Chebyshev features through parallel affine branches and a Hadamard product. This method produces frequency interactions explicitly, allowing a kernel MLP to map each normalized spectral coordinate to a complex channel‑mixing matrix while keeping the number of trainable parameters independent of the number of modes. Experiments on five PDE benchmarks and ablation studies demonstrate the effectiveness of the proposed basis configuration, multiplicative composition, and bandwidth learnability.

By Hyungjoon Juen, Minwoo Shin
arXiv Machine Learning
Oct 8

YANchor-4B: Effective Long-Horizon Reasoning in O(N) Time with O(1) Memory

YANchor-4B is a recurrent model that achieves long-horizon reasoning in linear time while using constant memory by preserving key information as anchors for later retrieval. It outperforms linear-time, constant-state baselines on challenging math benchmarks, achieving 82.93% mean pass@1 on AIME 2024–2026 and 63.64% on HMMT. The model also provides several-fold higher batched long-generation throughput than Transformer and hybrid baselines on H100, and demonstrates superior general-purpose capabilities across dozens of benchmarks.

By Huishan Ji, Hua Xu, Weiming Zhang, Qirui Ye
arXiv Machine Learning
Oct 8

Sparse Planning in Visual World Models via Cost Gradients

The paper introduces COSTGRAD, a training‑free, goal‑conditioned selector that ranks spatial tokens by the gradient norm of the planning cost, enabling sparse token usage in token‑based world models. On AdaLN‑conditioned predictors at 50% sparsity, COSTGRAD matches or surpasses full‑token planning on three of four continuous‑control benchmarks while delivering a 2.6× wall‑clock speedup per planning step. Combining token sparsity with reduced CEM search yields an overall ~5× speedup, though the method’s effectiveness depends on the underlying architecture, as shown by a failure mode with concat‑based models.

By Yingchen Xu, Edward Grefenstette
arXiv Machine Learning
Oct 8

Oscillatory Neural Dynamics over Sheaves

The paper introduces ONDA, a graph learning framework that uses operator‑valued information waves to enable effective long‑range propagation in graph neural networks. ONDA evolves stalk‑valued representations through second‑order dynamics governed by learned sheaf transport operators, combining wave‑like propagation with expressive local geometry. Experiments on challenging benchmarks show that ONDA consistently outperforms scalar wave propagation, diffusive sheaf baselines, and state‑of‑the‑art models, demonstrating the advantage of coupling wave dynamics with matrix‑valued transport.

By Jan-Willem Van Looy, Alessandro Trenta, Alessio Gravina, Alessio Borgi, Ferdinando Zanchetta, Pietro Li\`o, Davide Bacciu, Rita Fioresi
arXiv Computer Vision
Oct 8

Do Vision Models Learn Physical Constraints or Rendering Shortcuts? A Counterfactual Benchmark for Grounded Physical Consistency

The paper introduces a counterfactual benchmark to evaluate whether vision models understand physical constraints or merely learn rendering shortcuts. Using Mitsuba 3, it generates 5,500 synthetic images across 500 scene families, each with a clean image and ten violations (shadows, reflections, support, surface response, occlusion). Diagnostic baselines (LLaVA‑1.5‑7B, Qwen2.5‑VL‑7B, InternVL3.5‑8B) achieve 64.0–67.8 % macro‑F1 on a held‑out synthetic test set, but performance drops to 40.8 % for LLaVA‑1.5‑7B under an intervention shift, indicating reliance on rendering cues rather than true physical reasoning.

By M. Moein Esfahani, Sepehr Salem, Mohammed Alser, Vince Calhoun
arXiv Machine Learning
Oct 8

AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility

AutoAdapt is a modular framework that automatically discovers latent domains and trains per-domain Low‑Rank Adaptation (LoRA) adapters independently in parallel, without modifying other adapters. It uses parameter‑free routing to integrate new domains, achieving performance comparable to a LoRA adapter trained on all domains while avoiding full‑model retraining. The approach prevents domain interference by construction, enabling taxonomy‑free domain specialization across 14 benchmarks and GPT‑4o evaluations.

By Josh McGiff, Salma Mekaoui, Robert Shanahan, Nikola S. Nikolov
arXiv Machine Learning
Oct 8

ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals

ResidualQuant introduces a 2‑bit residual quantization scheme for KV caches in looped Transformers, leveraging the similarity of KV states across loops. By using the final‑loop KV states as a reference and applying least‑square scaling, rotations, and loop‑wise mixed precision, the method achieves accurate quantization down to INT2. Experiments on various looped Transformer models show that ResidualQuant improves the accuracy‑memory tradeoff, retaining near BF16 accuracy while reducing KV storage by 80.7% and boosting decode throughput and batch size on an RTX 5090.

By Heejun Kim, Junyoung Lee, SangLyul Cho, Dongsu Han, Insu Han, Sehoon Kim
arXiv Machine Learning
Oct 8

Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

The paper introduces the Expected Decoding Rounds (EDR) objective, a theoretical framework that models speculative decoding as a Markov reward process and directly optimizes the number of decoding rounds. EDR weights local rejection costs by state occupancies, eliminating auxiliary hyperparameters and enabling unbiased stochastic optimization via temporal‑difference gradients. Applying EDR to fine‑tune state‑of‑the‑art draft models (DSpark and DFly) improves mean accepted length and outperforms prior training objectives across nine benchmarks in math reasoning, code generation, and chat.

By Yunxiao Zhao, Changxiao Cai
arXiv Machine Learning
Oct 8

Physics-Aligned Electronic Ground-State Learning Improves Generalization

The paper introduces Physics‑Aligned Electronic Ground‑State Models (GSMs) that sit between machine‑learned interatomic potentials and Kohn‑Sham density functional theory in terms of computational cost and accuracy. By aligning GSM learning objectives with KS‑DFT equations and enforcing physical constraints, the authors develop new training techniques—OrthoNormal‑Loss (ON‑Loss), Grassmann Restricted Occupied‑Orbital Training (GROOT), and Residual Optimal‑gauge Conditioning‑aware KS‑Eq. Training (ROCKET)—which dramatically reduce energy and force errors in size‑extrapolation tests from QM9 to QM40. The models also achieve near‑chemical‑accuracy predictions on reactive chemistry benchmarks and demonstrate efficient label‑free self‑consistency fine‑tuning. whyItMatters":"The work advances the cost‑accuracy frontier for electronic structure predictions, enabling more reliable out‑of‑distribution generalization in drug and material discovery."

By Eike S. Eberhard, Xaver Kainz, Viktor Kotsev, Abdulrahman Aldossary, Stephan G\"unnemann
arXiv Machine Learning
Oct 8

Learning Transition Kernels of Jump-Diffusion Processes with Conditional Diffusion Models

The paper investigates learning transition kernels for time‑homogeneous jump‑diffusion processes using conditional diffusion models, aiming to generate new sample paths from high‑frequency discrete observations. It provides non‑asymptotic bounds on conditional score estimation error and KL divergence between true and generated path distributions. Experiments on synthetic data validate the theory and compare favorably to Gao et al. (2025), while real‑world applications demonstrate its utility for probabilistic forecasting.

By Yuzhen Zhao, Yating Liu, Quentin Guibert
arXiv Machine Learning
Oct 8

Learning Unknown Constraints without Unsafe Data via Optimality and Counterfactual Regularization

The paper introduces Counterfactual KKT (CF‑KKT), a framework that learns unknown constraints from locally optimal demonstrations without requiring known dynamics or risky exploration. CF‑KKT uses a learned differentiable dynamics model to apply KKT‑inspired optimality conditions and generates counterfactual behaviors that reveal infeasible data, enabling safer and more data‑efficient constraint recovery. Experiments on high‑dimensional robotic control tasks show that neural constraint representations learned with CF‑KKT achieve better safety and data efficiency than state‑of‑the‑art offline inverse constrained reinforcement learning baselines.

By Zhouyu Zhang, Chih-Yuan Chiu, Glen Chou
arXiv Machine Learning
Oct 8

Democratizing MoE inference on commodity GPUs with CoMoE

The paper introduces CoMoE, a communication‑efficient Mixture‑of‑Experts inference system designed for commodity GPUs. By using host‑centric routing, host‑backed token multicast, and fine‑grained token‑level aggregation, CoMoE reduces communication volume and eliminates global synchronization stalls. Experiments on RTX 5090 GPUs show up to 1.46× throughput improvement, approaching NVLink‑capable A800 performance while costing only 23.4% of the hardware.

By Ruwen Fan (Jimmy), Yuezhi Zu (Jimmy), Junru Li (Jimmy), Qingda Hu (Jimmy), Xinjun (Jimmy), Yang, Jiwu Shu, Youyou Lu
arXiv Machine Learning
Oct 8

Residual Learning in Empirical Asset Pricing

The paper demonstrates that residual learning enables neural network models in asset pricing to become deeper while preserving and refining shallow counterparts. Deep residual models achieve a higher out‑of‑sample Sharpe ratio (2.07) compared to shallow models (1.92) and significantly outperform deep feedforward models (0.89). This suggests that model depth can add economic value and offers a scalable approach for large asset pricing models.

By Dexin Peng, Xiaoyu Wang
arXiv Machine Learning
Oct 8

Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs

The paper introduces G^2MLP, a graph‑free MLP trained via distillation from a GNN teacher while preserving the teacher’s graph‑induced geometry. It identifies two spectral failure modes—underfit on sparse graphs and overfit on dense graphs—caused by neglecting geometry during distillation. By using Ollivier‑Ricci curvature to guide supervision between prediction‑level and representation‑level alignment, G^2MLP improves node‑classification performance and reduces the teacher‑student rank gap across benchmarks.

By Zhewei Chen, Hao Zhu, Jiaojiao Jiang, Ahad N. Zehmakan