Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,477 stories · RSS feed

arXiv AI

Jun 30

Mechanistic Personality Analysis of LLMs Steering Personality via Latent Feature Interventions

arXiv:2606. 28770v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated the ability to simulate human-like OCEAN personality traits in generated text.

By David Courtis, Ting Hu

llms fine-tuning benchmarks safety

arXiv AI

Jun 30

Model Merging to Evolution: Parameter Space Exploration for Expert Models

arXiv:2606. 28373v1 Announce Type: cross Abstract: Model merging integrates the capabilities of multiple expert models to create strong models for multiple tasks without additional training, thereby reducing computational resource requirements.

By Chao Wang, Yuchen Guo, Zheng Tan, Guanchun Wang, Yanbiao Ma, Qiqi Duan, Peng Wu

benchmarks

arXiv Machine Learning

Jun 30

Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries

arXiv:2602. 04940v2 Announce Type: replace Abstract: Deep learning has emerged as a transformative tool for the neural surrogate modeling of partial differential equations (PDEs), known as neural PDE solvers.

By Hang Zhou, Haixu Wu, Haonan Shangguan, Yuezhou Ma, Huikun Weng, Jianmin Wang, Mingsheng Long

llms benchmarks

arXiv AI

Jun 30

SurgVLA-Bench: Towards Evaluating Vision-Language-Action Models for Laparoscopic Surgical Robotics

arXiv:2606. 29247v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models represent a promising direction for embodied intelligence in surgical robotics.

By Jiashuo Sun, Yue He, Wenxuan Liu, Tao Mao, Jiazheng Wang, Xiang Chen, Min Liu

diffusion robotics multimodal benchmarks

arXiv AI

Jun 30

Beyond Triplet Plausibility: Relation Set Completion in Knowledge Graphs

arXiv:2606. 29860v1 Announce Type: new Abstract: Knowledge graphs (KGs) organize real-world knowledge as triplets and underpin many downstream applications.

By Zihao Zheng, Borui Cai, Yao Zhao, Keshav Sood, Yong Xiang

rag benchmarks

arXiv Machine Learning

Jun 30

Alternative Graph Neural Networks: Synergizing GEV Models and Deep Learning for Travel Mode Choice Modeling

arXiv:2509. 07123v2 Announce Type: replace-cross Abstract: Generalized extreme value models capture dependence among choice alternatives in discrete choice modeling, but require this dependence to be predefined, symmetric, and shared uniformly across individuals.

By Yuqi Zhou, Zhanhong Cheng, Dingyi Zhuang, Lingqian Hu, Yuheng Bu, Shenhao Wang

benchmarks

arXiv AI

Jun 30

ImprovEvolve: Basin-Hopping Meets LLM-Guided Evolutionary Search

arXiv:2602. 10233v2 Announce Type: replace-cross Abstract: LLM-guided evolutionary computation, most notably AlphaEvolve, has been remarkably successful in discovering novel mathematical constructions by solving challenging optimization problems.

By Alexey Kravatskiy, Valentin Khrulkov, Ivan Oseledets

llms benchmarks

arXiv AI

Jun 30

Online Data Selection for Instruction Tuning via Gaussian Processes

arXiv:2606. 30077v1 Announce Type: cross Abstract: With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic.

By Jun Wang, Quoc Phong Nguyen, Julien Monteil, Vu Nguyen

llms fine-tuning benchmarks data-engineering

arXiv AI

Jun 30

Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction

arXiv:2606. 29445v1 Announce Type: cross Abstract: Video understanding is a fundamental capability for multimodal intelligence, and recent Multimodal Large Language Models (MLLMs) have achieved remarkable performance on Video Question Answering (VideoQA) benchmarks.

By Sunqi Fan, Qingle Liu, Runqi Yin, Meng-Hao Guo, Shuojin Yang

llms agents nlp multimodal benchmarks

arXiv AI

Jun 30

Dynamic Parsing and Updating Natural Language Specification using VLMs for Robust Vision-Language Tracking

arXiv:2606. 29357v1 Announce Type: cross Abstract: Vision-language tracking guided by natural language specifications leverages high-level semantic cues of target objects to substantially boost tracking accuracy and robustness.

By Xiao Wang, Liye Jin, Dan Xu, Yuehang Li, Lan Chen, Yaowei Wang, Yonghong Tian, Jin Tang

llms multimodal benchmarks safety

arXiv AI

Jun 30

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).

By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao

llms fine-tuning benchmarks safety

arXiv AI

Jun 30

Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing

arXiv:2603. 03143v2 Announce Type: replace-cross Abstract: Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm.

By Jiyuan Wang, Chunyu Lin, Lei Sun, Zhi Cao, Yuyang Yin, Lang Nie, Zhenlong Yuan, Xiangxiang Chu, Yunchao Wei, Kang Liao, Guosheng Lin

diffusion computer-vision reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Models

arXiv:2604. 04385v5 Announce Type: replace-cross Abstract: We localize the policy routing mechanism in alignment-trained language models.

By Gregory N. Frank

llms benchmarks safety

arXiv Machine Learning

Jun 30

Solver-Integrated Adversarial Attacking and Training of Neural Operators

arXiv:2510. 18989v2 Announce Type: replace Abstract: Neural operators are commonly utilized as fast surrogates for numerical solvers in PDE problems, mapping input functions to solution functions.

By Yifei Sun

benchmarks safety

arXiv AI

Jun 30

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?

arXiv:2511. 06090v3 Announce Type: replace-cross Abstract: Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce runtime while preserving program correctness.

By Jeffrey Jian Ma, Milad Hashemi, Amir Yazdanbakhsh, Kevin Swersky, Ofir Press, Enhui Li, Vijay Janapa Reddi, Parthasarathy Ranganathan

llms agents benchmarks data-engineering

arXiv AI

Jun 30

When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate

arXiv:2512. 03578v3 Announce Type: replace-cross Abstract: Time series extrinsic regression (TSER) refers to the task of predicting a continuous target variable from an input time series.

By Florent Forest, Amaury Wei, Olga Fink

benchmarks safety

arXiv AI

Jun 30

ORCA: Open-ended Response Correctness Assessment for Audio Question Answering

arXiv:2512. 09066v2 Announce Type: replace-cross Abstract: Reliable assessment of the abilities of large audio language models (LALMs) is essential to advancing the state of the art.

By \v{S}imon Sedl\'a\v{c}ek, Sara Barahona, Bolaji Yusuf, Laura Herrera-Alarc\'on, Santosh Kesiraju, Cecilia Bola\~nos, Alicia Lozano-Diez, Sathvik Udupa, Fernando L\'opez, Allison Ferner, Ramani Duraiswami, Jan \v{C}ernock\'y

llms nlp benchmarks

arXiv AI

Jun 30

Demonstration-Free Robotic Control via LLM Agents

arXiv:2601. 20334v2 Announce Type: replace-cross Abstract: Robotic manipulation has increasingly adopted vision-language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift.

By Brian Y. Tsui, Alan Y. Fang, Tiffany J. Hwu

llms agents robotics fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 30

Momentum Guidance: Plug-and-Play Guidance for Flow Models

arXiv:2602. 20360v2 Announce Type: replace Abstract: Flow-based generative methods offer a simple and effective framework for high-fidelity generation, yet pretrained flow models are rarely used in their vanilla conditional form: in image generation, samples without guidance often appear diffuse and lack fine-grained detail.

By Runlong Liao, Jian Yu, Baiyu Su, Chi Zhang, Lizhang Chen, Qiang Liu

diffusion benchmarks

arXiv AI

Jun 30

SearchSkill: Teaching LLMs to Use Search Tools with Evolving Skill Banks

arXiv:2605. 09038v3 Announce Type: replace Abstract: Teaching language models to use search tools is not only a question of whether they search, but also of whether they issue good queries.

By Jinchao Hu, Meizhi Zhong, Kehai Chen, Min Zhang

llms nlp fine-tuning benchmarks

Get the daily briefing