Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv AI

Sep 15

Mesh-based Super-resolution of Multiscale Detonation Flows with Graph Transformers

arXiv:2511.12041v4 Announce Type: replace-cross Abstract: Super-resolution flow reconstruction using state-of-the-art data-driven techniques is valuable for a variety of applications, such as subgrid...

By Shivam Barwey, Pinaki Pal

llms benchmarks

arXiv AI

Sep 15

Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores

arXiv:2601.13885v2 Announce Type: replace-cross Abstract: Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluatio...

By Esma Balk{\i}r, Alice Pernthaller, Marco Basaldella, Jos\'e Hern\'andez-Orallo, Nigel Collier

llms rag benchmarks

arXiv AI

Sep 15

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

arXiv:2602.13576v2 Announce Type: replace-cross Abstract: Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-langu...

By Ruomeng Ding, Yifei Pang, He Sun, Yizhong Wang, Zhiwei Steven Wu, Zhun Deng

llms benchmarks safety

arXiv AI

Sep 15

Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating

arXiv:2603.11321v3 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards improves reasoning in large language models, yet on-policy learning often suffers from cold-st...

By Yuning Wu, Ke Wang, Haoran Liu, Chaoqun Jia, Devin Chen, Kai Wei

llms reinforcement-learning benchmarks

arXiv AI

Sep 15

GP-VM$\times$SMA: Benchmarking General-Purpose Vision Models and Specialized Architectures for 2D Medical Image Segmentation

arXiv:2603.13044v2 Announce Type: replace-cross Abstract: Medical image segmentation (MIS) is a fundamental component of computer-assisted diagnosis and clinical decision support. Over the past decad...

By Vanessa Borst, Anna Riedmann, Samuel Kounev

computer-vision benchmarks

arXiv AI

Sep 15

Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models

arXiv:2605.22902v2 Announce Type: replace-cross Abstract: Generative Vision-Language Models (VLMs) perform well on multimodal reasoning, but how visual inputs are transformed to text remains poorly u...

By Dimitrios Damianos, Leon Voukoutis, Georgios Skyrianos, Vassilis Katsouros, Georgios Paraskevopoulos

llms multimodal benchmarks safety

arXiv AI

Sep 15

Agentic AI for Gravitational Wave Data Analysis: A Head-to-Head Comparison of Coding Agents Executing a Matched Filter Pipeline on Einstein Telescope Simulated Data

arXiv:2605.28916v3 Announce Type: replace-cross Abstract: We report a methodological study of agentic AI in gravitational-wave data analysis: two systems, Claude Code (Anthropic) and Codex (OpenAI),...

By Gianluca Inguglia

llms agents benchmarks

arXiv AI

Sep 15

Syn2Logic: End-to-End Neuromorphic Design Automation

arXiv:2608.25536v2 Announce Type: replace-cross Abstract: In this work, we propose a view on electronic Neuromorphic Design Automation (eNDA), which we see as a design automation flow that bridges co...

By Artur Podobas

benchmarks

arXiv Machine Learning

Sep 15

A derivative-fidelity failure mode in physics-informed neural networks: strengthened benchmark evidence from function-value training

arXiv:2609.13171v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) use automatic differentiation to impose differential-equation residuals, but good agreement in function values...

By Koji Koyamada

benchmarks

arXiv Machine Learning

Sep 15

Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation

arXiv:2609.13199v1 Announce Type: new Abstract: Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher mo...

By Dawen Jiang, Zhishu Shen, Zeyu Liu, Tiehua Zhang

efficiency benchmarks safety

arXiv Machine Learning

Sep 15

Pretraining for Sample-Efficient Neural Interfaces

arXiv:2609.13507v1 Announce Type: new Abstract: Brain-computer interfaces (BCIs) decode neural activity to restore lost function. Typically, training a high-performance neural decoder requires a larg...

By Ben Tang, Zachary Spalding, Gregory B. Cogan

rag fine-tuning benchmarks

arXiv Machine Learning

Sep 15

A Multi-Resolution Multi-Domain Pre-Training Framework for Universal Traffic Forecasting

arXiv:2609.13878v1 Announce Type: new Abstract: Spatio-temporal traffic data are central to intelligent transportation systems, yet their heterogeneity poses significant challenges for large-scale mo...

By Zhouyang Liu, Jindong Han, Hao Wang, Xinyue Liu, Hui Gao, Dongsheng Li, Hao Liu

diffusion benchmarks

arXiv Machine Learning

Sep 15

Draining Fictitious Knots: Restoring Distance-Awareness Guarantees for High-Dimensional Spline Networks

arXiv:2609.15274v1 Announce Type: new Abstract: Kolmogorov-Arnold Networks (KANs) with spline activations have recently shown promise for interpretable function approximation. Distance-Aware Error fo...

By Masoud Ataei, Mohammad Javad Khojasteh, Vikas Dhiman

benchmarks

arXiv Machine Learning

Sep 15

Survey of Novel Deep Learning Architectures for Denoising Gravitational-wave Signals

arXiv:2609.13272v1 Announce Type: cross Abstract: Gravitational-wave denoising must handle the full diversity of spinning, precessing binaries, since the recovered waveform underpins parameter estima...

By Rohan Raha, Prayush Kumar

benchmarks

arXiv Machine Learning

Sep 15

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

arXiv:2609.13285v1 Announce Type: cross Abstract: The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length. Grouped-query a...

By Vishesh Tripathi, Abhay Kumar, Ramsha Khan

llms efficiency benchmarks

arXiv Machine Learning

Sep 15

Fusing Spectral Signatures and Activation Clustering for Backdoor Detection in Healthcare Imaging Models: Method, Implementation, and Evaluation

arXiv:2609.14290v1 Announce Type: cross Abstract: Machine learning models are increasingly deployed in healthcare imaging pipelines for diagnostic support, and training-time attacks against them are...

By Suresh Tamang

benchmarks safety

arXiv Machine Learning

Sep 15

Artificial entrepreneurial cognition: Locating and causally steering an opportunity recognition dial inside large language models (LLMs)

arXiv:2609.15277v1 Announce Type: cross Abstract: Entrepreneurial cognition is a foundation of entrepreneurship research. Yet the growing involvement of large language models (LLMs) in entrepreneuria...

By Christian Fisch, Angela Altmeier, Martin Obschonka, Michal Kosinski, Pin Ni

llms robotics benchmarks safety

arXiv AI

Sep 15

CodeTS: Verifiable Text-to-Time Series Generation via Executable Code

CodeTS introduces a verifiable framework for generating time series from natural language by translating textual temporal descriptions into executable code, which then produces the desired series. The approach constructs aligned Text‑Code‑TS triplets for supervised initialization and employs multi‑stage execution‑based rewards to ensure code validity and time‑series quality. Experiments on eight benchmarks show that CodeTS outperforms both LLM‑based and supervised generative baselines, offering a strong zero‑shot solution for Text‑to‑TS generation.

By Xudong Yuan, Shunyu Liu, Tongya Zheng, Huiping Zhuang, Mingli Song, Kaixuan Chen

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 15

Realistic Continual Learning Approach using Pre-trained Models

arXiv:2404.07729v2 Announce Type: replace Abstract: Continual learning (CL) evaluates adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forg...

By Nadia Nasri, Carlos Guti\'errez-\'Alvarez, Sergio Lafuente-Arroyo, Saturnino Maldonado-Basc\'on, Roberto J. L\'opez-Sastre

benchmarks

arXiv Machine Learning

Sep 15

CLIMATEAGENT: Multi-Agent Orchestration for Complex Climate Data Science Workflows

arXiv:2511.20109v2 Announce Type: replace Abstract: Climate science demands automated workflows to transform comprehensive questions into data-driven statements across massive, heterogeneous datasets...

By Chenyue Li, Hyeonjae Kim, Wen Deng, Mengxi Jin, Wen Huang, Mengqian Lu, Binhang Yuan

llms agents benchmarks

Get the daily briefing