Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv AI

Jun 4

ANN Search: Recall What Matters

arXiv:2606. 04522v1 Announce Type: cross Abstract: Approximate nearest neighbor (ANN) search has become a core primitive in information retrieval and modern machine learning tasks, from classification to retrieval-augmented generation.

By Dimitris Dimitropoulos, Nikos Mamoulis

llms rag efficiency benchmarks

arXiv AI

Jun 4

Curvature-aware dynamic precision approach for physics-informed neural networks

arXiv:2606. 04736v1 Announce Type: cross Abstract: Physics-informed neural networks (PINNs) have become a promising framework for simulating partial differential equations (PDEs) by embedding physical laws directly into neural network training.

By Yingjie Shao, Ioannis N. Athanasiadis, George van Voorn, Taniya Kapoor

rag benchmarks

arXiv AI

Jun 4

Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control

arXiv:2606. 04775v1 Announce Type: cross Abstract: Text-to-video (T2V) models trained on large-scale web data can generate undesired content, motivating interventions that reduce harmful outputs without sacrificing visual quality.

By Jihoon Hong, Alice Chan, Qiyue Dai, Julian Skifstad, Glen Chou

diffusion fine-tuning benchmarks safety

arXiv AI

Jun 4

Coarse-to-fine Hierarchical Architecture with Sequential Mamba for Brain Reconstruction

arXiv:2606. 04772v1 Announce Type: cross Abstract: Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience.

By Hoang-Son Vo, Van-Hung Bui, Minh-Huy Mai-Duc, Tien-Dung Mai, Soo-Hyung Kim

computer-vision benchmarks

arXiv AI

Jun 4

Geometry-Aware Distillation for Prompt Tuning Biomedical Vision-Language Models

arXiv:2606. 04922v1 Announce Type: cross Abstract: Current prompt-based and adapter-based tuning of vision-language models (VLMs) is attractive for medical imaging, where clinical data sensitivity favors frozen backbones and annotations are limited.

By Tran Dinh Tien, Zhiqiang Shen

llms efficiency multimodal benchmarks safety

arXiv Machine Learning

Jun 4

Neural Variance-aware Dueling Bandits with Deep Representation and Shallow Exploration

arXiv:2506. 01250v3 Announce Type: replace Abstract: We introduce the first variance-aware algorithms for contextual dueling bandits that leverage shallow exploration strategies with neural networks for nonlinear utility approximation.

By Youngmin Oh, Jinje Park, Taejin Paik

benchmarks

arXiv AI

Jun 4

Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recovery

arXiv:2606. 05037v1 Announce Type: cross Abstract: When an AI agent calls an API and hits a validation error, it needs more than what went wrong -- it needs what to do next.

By Arquimedes Canedo, Grama Chethan

llms agents benchmarks safety

arXiv AI

Jun 4

M$^3$Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

arXiv:2606. 05008v1 Announce Type: cross Abstract: As multi-modal models advance towards long-form video understanding, memory emerges as a critical capability.

By Jie Huang, Ruixun Liu, Sirui Sun, Xinyi Yang, Yin Li, Yixin Zhu, Yiwu Zhong

multimodal benchmarks

arXiv AI

Jun 4

Invariant Gradient Alignment for Robust Reasoning Distillation

arXiv:2606. 05025v1 Announce Type: cross Abstract: Large language models (LLMs) suffer from shortcut learning: they systematically fail on out-of-distribution (OOD) inputs whose semantic surface differs from training data, even when the logical structure is identical.

By Zehua Cheng, Wei Dai, Jiahao Sun

llms fine-tuning efficiency benchmarks safety

arXiv AI

Jun 4

Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?

arXiv:2506. 10912v4 Announce Type: replace Abstract: Toxicity remains a leading cause of early-stage drug development failure.

By Fei Lin, Ziyang Gong, Cong Wang, Tengchao Zhang, Yonglin Tian, Yining Jiang, Ji Dai, Chao Guo, Xiaotong Yu, Xue Yang, Gen Luo, Fei-Yue Wang

llms efficiency multimodal benchmarks safety

arXiv AI

Jun 4

Adaptive Minds: Empowering Agents with LoRA-as-Tools

arXiv:2510. 15416v2 Announce Type: replace Abstract: We investigate a framework in which LoRA adapters are treated as callable tools that a base language model can dynamically select and invoke.

By Pavan C Shekar, Aswanth Krishnan

llms agents fine-tuning benchmarks

arXiv AI

Jun 4

SciDER: Scientific Data-centric End-to-end Researcher

arXiv:2603. 01421v3 Announce Type: replace Abstract: While large language models accelerate scientific discovery, existing agents face severe limitations in adaptability, domain generalization, and multimodal scalability, often struggling to autonomously process raw, domain-specific experimental data.

By Ke Lin, Owais Aijaz, Yilin Lu, Yiyang Luo, Xuehang Guo, Preslav Nakov

llms agents fine-tuning multimodal benchmarks

arXiv AI

Jun 4

Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time

arXiv:2504. 12329v2 Announce Type: replace-cross Abstract: Recent advances leverage post-training to enhance model reasoning performance, which typically requires costly training pipelines and still suffers from inefficient, overly lengthy outputs.

By Wang Yang, Xiang Yue, Vipin Chaudhary, Xiaotian Han

benchmarks

arXiv AI

Jun 4

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training

arXiv:2506. 05233v2 Announce Type: replace-cross Abstract: Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention.

By Johannes von Oswald, Nino Scherrer, Seijin Kobayashi, Luca Versari, Songlin Yang, Sarthak Mittal, Maximilian Schlegel, Kaitlin Maile, Yanick Schimpf, Oliver Sieberling, Alexander Meulemans, Rif A. Saurous, Guillaume Lajoie, Charlotte Frenkel, Razvan Pascanu, Blaise Ag\"uera y Arcas, Jo\~ao Sacramento

llms benchmarks

arXiv AI

Jun 4

ClustRecNet: A Novel End-to-End Deep Learning Framework for Clustering Algorithm Recommendation

arXiv:2509. 25289v4 Announce Type: replace-cross Abstract: Identifying an effective clustering algorithm for a given dataset remains a fundamental unsupervised learning issue.

By Mohammadreza Bakhtyari, Bogdan Mazoure, Renato Cordeiro de Amorim, Guillaume Rabusseau, Vladimir Makarenkov

benchmarks

arXiv AI

Jun 4

BioBlue: Systematic runaway-optimiser-like LLM failure modes on biologically and economically aligned AI safety benchmarks for LLMs with simplified observation format

arXiv:2509. 02655v3 Announce Type: replace-cross Abstract: Many AI alignment discussions of "runaway optimisation" focus on RL agents: unbounded utility maximisers that over-optimise a proxy objective (e.

By Roland Pihlakas (for the Three Laws collaboration), Sruthi Susan Kuriakose (for the Three Laws collaboration)

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 4

Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models

arXiv:2601. 09719v3 Announce Type: replace-cross Abstract: Pre-Layer Normalization (Pre-LN) is the de facto choice for large language models (LLMs) and is crucial for stable pretraining and effective transfer learning.

By Hoyoon Byun, Youngjun Choi, Taero Kim, Sungrae Park, Kyungwoo Song

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 4

AlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithms

arXiv:2602. 09464v2 Announce Type: replace-cross Abstract: Vericoding refers to the generation of formally verified code from rigorous specifications.

By Haoyu Zhao, Ziran Yang, Jiawei Li, Deyuan He, Zenan Li, Chi Jin, Venugopal V. Veeravalli, Aarti Gupta, Sanjeev Arora

llms benchmarks

arXiv AI

Jun 4

Tomography by Design: An Algebraic Approach to Low-Rank Quantum States

arXiv:2602. 15202v2 Announce Type: replace-cross Abstract: We present an algebraic algorithm for quantum state tomography that leverages measurements of certain observables to estimate structured entries of the underlying density matrix.

By Shakir Showkat Sofi, Charlotte Vermeylen, Lieven De Lathauwer

benchmarks

arXiv AI

Jun 4

Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction

arXiv:2602. 23312v3 Announce Type: replace-cross Abstract: Leader-follower interaction is an important paradigm in human-robot interaction (HRI).

By Rafael R. Baptista, Andr\'e de Lima Salgado, Ricardo V. Godoy, Marcelo Becker, Thiago Boaventura, Gustavo J. G. Lahr

llms robotics fine-tuning benchmarks

Get the daily briefing