Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,620 stories · RSS feed

arXiv AI

Sep 10

SkillAdam: Stable and Efficient Skill Evolution for Agents

arXiv:2609.08944v1 Announce Type: new Abstract: Agent skills provide a lightweight way to equip frozen language-model agents with domain knowledge and procedural guidance, yet obtaining high-quality...

By Gaoyuan Li, Meihao Fan, Yizhe Liu, Shaolei Zhang, Ju Fan, Siyi Wang, Jiaheng Hou, Xudong Weng, Honghan Tian, Zang Li

agents benchmarks

arXiv Machine Learning

Sep 10

Differentiable Causal Discovery for Singular Linear Models under Confounding

arXiv:2601.01368v2 Announce Type: replace Abstract: Score-based causal discovery in the presence of unobserved confounders requires both a consistent scoring criterion and an efficient search over gr...

By Mujin Zhou, Ignavier Ng, Junzhe Zhang

benchmarks

arXiv AI

Sep 10

SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement

arXiv:2609.05850v1 Announce Type: cross Abstract: Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has reve...

By Quoc Viet Vo, Trung Le, Damith C. Ranasinghe, Ehsan Abbasnejad

llms benchmarks safety

arXiv Computer Vision

Sep 10

Morphological Decoupling-Based Skeletal Classification for Clinical Assessment of Malocclusion

arXiv:2609.09801v1 Announce Type: cross Abstract: Malocclusion skeletal grading is a fundamental task in orthodontics, critical for diagnosis and treatment planning. Traditionally, cone-beam computed...

By Zhichun Jin, Zhicheng He, Hao Xu, Dongyang Li, Lin Wang, Hongliang Ren, Long Bai

multimodal benchmarks safety

arXiv Computer Vision

Sep 10

AVSRBench: A Multi-Condition AVSR Benchmark

arXiv:2609.10366v1 Announce Type: cross Abstract: While AVSR has achieved sub-1% word error rates on the standard LRS3 benchmark, its reliance on broadcast speech obscures whether this reflects true...

By Rishabh Jain, Naomi Harte

llms fine-tuning multimodal benchmarks

arXiv AI

Sep 10

Adversarial Resilience of Poisson-Process Submodular Maximization over Matroids, and Full-Bandit Learning

arXiv:2608.12134v2 Announce Type: replace-cross Abstract: We study nonnegative submodular maximization on $n$ elements subject to a general matroid of rank $k$, when the offline algorithm is given an...

By Vaneet Aggarwal

reinforcement-learning benchmarks safety

arXiv AI

Sep 10

UniRRM: Unified Reasoning Reward Models Across Languages and Evaluation Paradigms

arXiv:2609.05910v1 Announce Type: cross Abstract: Reinforcement learning (RL) excels on tasks with verifiable rewards, but in open-ended tasks, the reliability of reward models remains a key challeng...

By Peng Lai, Yichao Du, Junchao Wu, Weibo Gao, Linan Yue, Longyue Wang, Weihua Luo, Derek F. Wong, Guanhua Chen

llms reinforcement-learning benchmarks safety

arXiv AI

Sep 10

STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models

arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...

By Yichen Guo, Tinghao Wang, Qizhe Zhang, Lingbei Meng, Yuan Zhang, Jiajun Cao, Hao Jiang, Chenwei Wu, Jixian Wu, Sixiang Chen, Tao Luo, Hongyang Cheng, Kai Tang, Chenxi Li, Renyuan Li, Xiande Huang, Wenya Wang, Shanghang Zhang

llms efficiency multimodal benchmarks

arXiv AI

Sep 10

VST: Verifiable Structured Transport for Auditable Agent-to-Agent Alpha Discovery

arXiv:2609.07065v1 Announce Type: new Abstract: Agent-to-agent (A2A) alpha discovery is slowed by repeated feedback cycles between mining and evaluation agents, whose hand-offs, in contemporary LLM m...

By Yuqi Li, Siyuan Liu, Bingjun Liu

llms agents benchmarks

arXiv AI

Sep 10

Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems

arXiv:2609.05928v1 Announce Type: cross Abstract: Large language models now compute correct tax liabilities on over 90% of well-formed cases in statutory benchmarks, which makes them candidates for t...

By Albert Sadowski, Jaros{\l}aw A. Chudziak

llms benchmarks

arXiv AI

Sep 10

Risk Is Not Review Value: Wrong-Answer Exposure Under Bounded Review Budgets

arXiv:2609.07095v1 Announce Type: new Abstract: LLM assistants often produce more answers than humans can review before users see them. Most evaluations ask whether an answer is wrong, unsupported, o...

By SangJin Park, Myungsub Choi, Jineok Kim, Minseung Kang

llms benchmarks

arXiv AI

Sep 10

When Intelligence Becomes Agency: A Theory of Governed, Proactive Agency for Symbiotic AI Systems

arXiv:2609.07741v1 Announce Type: new Abstract: Persistent AI assistants are intended to extend human attention, memory, and coordination across changing digital and physical environments. To be trul...

By Jo\~ao Dias Ferreira

agents robotics benchmarks

arXiv AI

Sep 10

CausalVerify: An Execution-Grounded Benchmark for LLM Causal Inference Workflows

arXiv:2609.07944v1 Announce Type: new Abstract: Existing causal-inference benchmarks for LLMs mostly score method descriptions or whether generated code runs, not whether the executed workflow recove...

By Yonghong Zhang, Ricardo Correia, Isabel M. Parra, Yong Xie

llms benchmarks

arXiv Machine Learning

Sep 10

On-the-go Forgetting without Explicit Unlearning via ERASE

arXiv:2609.05966v1 Announce Type: new Abstract: Existing unlearning approaches typically rely on post hoc weight adaptation or distillation, leading to duplicated memory costs, degraded generalizatio...

By Kushal Chakrabarti, Mayank Baranwal

fine-tuning efficiency benchmarks safety

arXiv AI

Sep 10

Automated Design of Inventory Policy with Large Language Models: An Exploratory Study

arXiv:2609.08071v1 Announce Type: new Abstract: Firms making inventory decisions have access to operational data, optimization tools, and large language models (LLMs). Typically, data characterize th...

By Fenghua Yang, Preet Baxi, Yi Zhang, Stefanus Jasin, Yanzhe Lei, Mo Liu, Parshan Pakiman

llms benchmarks

arXiv Machine Learning

Sep 10

FANS: Federated Adaptive Network Search Learning for Heterogeneous Devices

arXiv:2609.06106v1 Announce Type: new Abstract: Heterogeneous Federated Learning (HFL) aims to train models across devices with diverse resource budgets while preserving data privacy. Existing HFL me...

By Jiaxin Zhang, Xingwei Wang, Bo Yi, Liang Zhao, Alireza Furutanpey, Ziyi Chen, Qiang He, Keqin Li, Schahram Dustdar

efficiency benchmarks safety

arXiv Machine Learning

Sep 10

DataFlex-RL: An Evaluation Platform for RLVR Data Policies

arXiv:2609.06107v1 Announce Type: new Abstract: Data policies for reinforcement learning with verifiable rewards (RLVR) determine which rollouts are used, how strongly they are weighted, and which do...

By Hao Liang, Mingrui Chen, Hengyi Feng, Meiyi Qiang, Wentao Zhang

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 10

Sector-Mean: Deterministic Initialization of K-Means Centroids via Angular Sector Partitioning

arXiv:2609.06468v1 Announce Type: new Abstract: K-Means is one of the most widely used clustering algorithms, but its susceptibility to initial centroid selection remains a primary bottleneck for its...

By Abhiyan Dhakal (Kathmandu University), Pranish Kafle (Kathmandu University), Rajani Chulyadyo (Kathmandu University)

benchmarks

arXiv AI

Sep 10

Do Dynamic Routers Need Memory? HeRo: History-Aware Routing for Efficient LLM Inference

arXiv:2609.08189v1 Announce Type: new Abstract: Dynamic layer routing reduces the inference cost of Large Language Models (LLMs) by learning to skip layers for individual tokens. Existing methods, ho...

By Hongjin Lin, Wentao Wan, Keze Wang

llms benchmarks

arXiv Statistics ML

Sep 10

DFNN: A Deep Fr\'echet Neural Network Framework for Learning Metric-Space-Valued Responses

arXiv:2510.17072v2 Announce Type: replace Abstract: Regression with non-Euclidean responses---e.g., probability distributions, networks, symmetric positive-definite matrices, and compositions---has b...

By Kyum Kim, Yaqing Chen, Paromita Dubey

benchmarks

Get the daily briefing