Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,930 stories · RSS feed

arXiv AI

Jun 30

Recursive Self-Evolving Agents via Held-Out Selection

arXiv:2606. 28374v1 Announce Type: new Abstract: LLM agents are increasingly improved without weight updates by evolving a natural-language artifact, such as reflections, workflows, playbooks, cheatsheets, or optimized prompts, that conditions a frozen policy.

By Michael Nguyen, Quoc Nguyen, Paul Vuong

llms agents benchmarks

arXiv AI

Jun 30

Data and Evaluation Closed-Loop for Model Capability Enhancement

arXiv:2606. 28471v1 Announce Type: new Abstract: Model capability is the central variable in LLM pre-training, yet is never observed directly: data shapes it prospectively, while evaluation reveals it only retrospectively, compressing samples, prompts, decoding, and scoring rules into one noisy score.

By Zhixuan Li, Jiangan Yuan, Han Xu

llms benchmarks

arXiv Machine Learning

Jun 30

Boundary Degree as a Node-level Feature for Epidemic Scenario Identification in Agent-based Cascade Simulations

arXiv:2606. 29596v1 Announce Type: cross Abstract: Characterizing the scenario underlying an epidemic from its disease cascade is an important task in simulation analytics.

By Amro Alabsi Aljundi, Galen Harrison, Jiangzhuo Chen, Abhijin Adiga, Anil Kumar Vullikanti, Madhav V. Marathe

agents benchmarks

arXiv Machine Learning

Jun 30

An Integrated Two-Stage Deep-Learning Tool for Rapid Post-Hurricane Damage Identification and Repair Scheduling

arXiv:2606. 29117v1 Announce Type: cross Abstract: Post-hurricane damage assessment and repair scheduling can require computationally intensive simulation and optimization.

By Hooman Torkaman, Ellis Oti Boateng, Jignesh Solanki, Anurag Srivastava

llms benchmarks

arXiv AI

Jun 30

Dynamic Parsing and Updating Natural Language Specification using VLMs for Robust Vision-Language Tracking

arXiv:2606. 29357v1 Announce Type: cross Abstract: Vision-language tracking guided by natural language specifications leverages high-level semantic cues of target objects to substantially boost tracking accuracy and robustness.

By Xiao Wang, Liye Jin, Dan Xu, Yuehang Li, Lan Chen, Yaowei Wang, Yonghong Tian, Jin Tang

llms multimodal benchmarks safety

arXiv AI

Jun 30

UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation

arXiv:2606. 29502v1 Announce Type: new Abstract: Skill memories can improve agentic reinforcement learning by reusing past experience as textual guidance, but retrieved skills are not oracular: they may help in one state while misleading the same policy in another.

By Songjun Tu, Chengdong Xu, Qichao Zhang, Yiwen Ma, Yaocheng Zhang, Linjing Li, Dong Li, Xiangyuan Lan, Dongbin Zhao

agents reinforcement-learning efficiency benchmarks

arXiv AI

Jun 30

SurgVLA-Bench: Towards Evaluating Vision-Language-Action Models for Laparoscopic Surgical Robotics

arXiv:2606. 29247v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models represent a promising direction for embodied intelligence in surgical robotics.

By Jiashuo Sun, Yue He, Wenxuan Liu, Tao Mao, Jiazheng Wang, Xiang Chen, Min Liu

diffusion robotics multimodal benchmarks

arXiv AI

Jun 30

From Detecting Agency to Doing Work: Self-Caused Credit Builds a Durable Behavioral Self in a Minimal Spiking Agent

arXiv:2606. 30191v1 Announce Type: new Abstract: How does an agent that can tell self from world come to be durably shaped by that distinction?

By Haoliang Han

agents benchmarks

arXiv AI

Jun 30

Online Data Selection for Instruction Tuning via Gaussian Processes

arXiv:2606. 30077v1 Announce Type: cross Abstract: With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic.

By Jun Wang, Quoc Phong Nguyen, Julien Monteil, Vu Nguyen

llms fine-tuning benchmarks data-engineering

arXiv AI

Jun 30

Ensemble Learning Based Classification Algorithm Recommendation

arXiv:2101. 05993v2 Announce Type: replace-cross Abstract: Selecting an appropriate classification algorithm for a given data set remains a challenging problem in data mining and machine learning.

By Guangtao Wang, Qinbao Song, Xiaoyan Zhu, Jiao Liu

benchmarks

arXiv AI

Jun 30

ImprovEvolve: Basin-Hopping Meets LLM-Guided Evolutionary Search

arXiv:2602. 10233v2 Announce Type: replace-cross Abstract: LLM-guided evolutionary computation, most notably AlphaEvolve, has been remarkably successful in discovering novel mathematical constructions by solving challenging optimization problems.

By Alexey Kravatskiy, Valentin Khrulkov, Ivan Oseledets

llms benchmarks

arXiv Machine Learning

Jun 30

Geometrically Principled Randomized Optimization for Efficient LLM Training

arXiv:2510. 01878v2 Announce Type: replace Abstract: Low-rank gradient optimization for large language models is currently divided into two categories: structured methods that rigorously identify subspaces, and randomized approaches employed primarily for computational efficiency.

By Sahar Rajabi, Nayeema Nonta, Sirisha Rambhatla

llms benchmarks

arXiv AI

Jun 30

Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents

arXiv:2606. 30185v1 Announce Type: new Abstract: Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools.

By Yutao Sun, Yanting Miao, Hao-Xuan Ma, Mengyu Zhou, Mingshuai Chen, Tiancheng Zhao, Dexin Wang, Lei Lv, Li Xu, Xiaoxi Jiang, Guanjun Jiang

llms agents reinforcement-learning multimodal benchmarks

arXiv AI

Jun 30

Beyond Triplet Plausibility: Relation Set Completion in Knowledge Graphs

arXiv:2606. 29860v1 Announce Type: new Abstract: Knowledge graphs (KGs) organize real-world knowledge as triplets and underpin many downstream applications.

By Zihao Zheng, Borui Cai, Yao Zhao, Keshav Sood, Yong Xiang

rag benchmarks

arXiv AI

Jun 30

Model Merging to Evolution: Parameter Space Exploration for Expert Models

arXiv:2606. 28373v1 Announce Type: cross Abstract: Model merging integrates the capabilities of multiple expert models to create strong models for multiple tasks without additional training, thereby reducing computational resource requirements.

By Chao Wang, Yuchen Guo, Zheng Tan, Guanchun Wang, Yanbiao Ma, Qiqi Duan, Peng Wu

benchmarks

arXiv AI

Jun 30

Flow Matching in Feature Space for Stochastic World Modeling

arXiv:2606. 29059v1 Announce Type: cross Abstract: World modeling requires forecasting uncertain futures while preserving information useful for downstream perception.

By Francois Porcher, Nicolas Carion, Karteek Alahari, Shizhe Chen

diffusion multimodal benchmarks

arXiv AI

Jun 30

Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models

arXiv:2606. 29892v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become indispensable for pushing Vision-Language-Action Models (VLAs) beyond static imitation learning.

By Siyao Chen, Jiakang Yuan, Jiaxin Wang, Tao Chen

reinforcement-learning multimodal benchmarks

arXiv AI

Jun 30

Dockerless: Environment-Free Program Verifier for Coding Agents

arXiv:2606. 28436v1 Announce Type: cross Abstract: Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning (RL).

By Wenhao Zeng, Yuling Shi, Xiaodong Gu, Chao Hu, Chaofan Wang, Yuhao Cui, Hongting Zhou, Mengnan Qi, Jianqiao Wangni, Zhaojian Yu, Shuzheng Gao, Kai Cai, Shilin He

agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs

arXiv:2606. 28438v1 Announce Type: cross Abstract: Recursive self-training can degrade neural generative models when generated data is reused without fresh human data or external quality control.

By Xinyuan Song, Zekun Cai, Liang Zhao

llms fine-tuning benchmarks

arXiv AI

Jun 30

Reinforcement Learning for Software Vulnerability Analysis: A Systematic Review with Emphasis on C/C++ Source Code and Static Analysis

arXiv:2606. 28403v1 Announce Type: cross Abstract: Vulnerability detection in C/C++ software remains a major security challenge due to code complexity, manual memory management, and the limitations of traditional static analysis.

By Bruno Caro-V\'asquez, Carola Figueroa-Flores, Gast\'on Marquez

agents reinforcement-learning benchmarks

Get the daily briefing