Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

14,907 stories · RSS feed

arXiv Machine Learning
Aug 4

Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment

arXiv:2605. 15720v2 Announce Type: replace-cross Abstract: Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly.

By Yuchen Li, Ziru Wei, Zhen Zhao, Yi Liu, Luping Zhou
arXiv Machine Learning
Aug 4

Capability Provenance in Language Models: A Case Study in Social Reasoning

arXiv:2606. 19625v2 Announce Type: replace-cross Abstract: We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-reasoning versus STEM-reasoning in OLMo3-7B.

By Glenn Matlin, Chandreyi Chakraborty, Saehee Eom, Mika Okamoto, Rayan Castilla, Louis Jaburi, Alvin Deng, Taywon Min, Lucia Quirke, Stella Biderman, Mark Riedl
arXiv Machine Learning
Aug 4

Self-Improving Large Language Models via Progressive Experience Evolution

arXiv:2608. 02139v1 Announce Type: cross Abstract: Large language models (LLMs) capable of self-improvement require not only effective policy optimization, but also a principled mechanism for transforming transient interaction experience into persistent model capabilities.

By Shijie Ren, Xiting Wang, Meng Li, Yujie Guo, Yunhang Yao, Ziheng Peng, Xunlong Wang, Yuetan Chen, Haoyang Zhou, Yunlong Liang, Fandong Meng
arXiv Machine Learning
Aug 4

Learned Digital Over-the-Air Computing for Federated Edge Learning

arXiv:2509. 16577v2 Announce Type: replace Abstract: Over-the-air (OTA) aggregation enables federated edge learning (FEEL) by exploiting the superposition property of the wireless channel to merge communication with computation, eliminating the need to schedule and decode devices individually.

By Antonio Tarizzo, Mohammad Kazemi, Deniz G\"und\"uz