Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv AI

Jul 16

ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation

arXiv:2607. 13124v1 Announce Type: cross Abstract: Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints can collapse on the free-form generation that deployment actually requires.

By Qingyu Zhang, Qianhao Yuan, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun, Xiang Li, Ming Xu, Jiarui Li, Xiuyin Zhao

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 16

Traffic-Aware Randomized Smoothing for LLM-Based Network Intrusion Detection

arXiv:2607. 13801v1 Announce Type: cross Abstract: Large language model (LLM)-based intrusion detection systems (IDS) are increasingly studied for security monitoring, yet their robustness against feasible traffic manipulation remains largely empirical.

By Zhenpeng Li

llms robotics fine-tuning benchmarks safety

arXiv AI

Jul 16

Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

arXiv:2607. 14004v1 Announce Type: new Abstract: Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method.

By Wenxiao Wang, Priyatham Kattakinda, Soheil Feizi

agents benchmarks safety

arXiv AI

Jul 16

ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding

arXiv:2607. 13421v1 Announce Type: cross Abstract: Spatio-Temporal Video Grounding (STVG) aims to retrieve the visual trajectory of a specific object from a video stream as described by a natural language expression.

By Kai Chen, Ming Dai, Wenxuan Cheng, Wankou Yang

multimodal benchmarks

arXiv Machine Learning

Jul 16

A plug-and-play approach with fast uncertainty quantification for weak lensing mass mapping

arXiv:2603. 22006v2 Announce Type: replace-cross Abstract: Upcoming stage-IV surveys such as Euclid and Rubin will deliver vast amounts of high-precision data, opening new opportunities to constrain cosmological models with unprecedented accuracy.

By Hubert Leterme, Andreas Tersenov, Jalal Fadili, Jean-Luc Starck

benchmarks

arXiv Machine Learning

Jul 16

DreamSat-Pose: Spacecraft Pose Estimation from Single-View 3D Reconstructions and Learned 2D-3D Feature Matching

arXiv:2607. 13449v1 Announce Type: cross Abstract: 6-DoF pose estimation is a critical task in autonomous rendezvous and proximity operations.

By Josiane Uwumukiza, Jocelyn Zhao, Giovanni Lavezzi, Giacomo Battaglia, Paolo Panicucci, Minduli C. Wijayatunga, Victor Rodriguez-Fernandez, Richard Linares

llms agents computer-vision benchmarks

arXiv AI

Jul 16

Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

arXiv:2607. 13125v1 Announce Type: cross Abstract: We introduce Boogu-Image-0.

By Guoxuan Chen, Chufeng Xiao, Haoran Yang, Siyue Xie, Binxiao Huang, Ming Zhang, Cheuk Him Chau, Xinyu Fu, Yingzhao Lian, Tom S. Y. Li, Jintao Lin, Bowen Dong, Zian Qian, Yuhao Liu, Yuxuan Hu, Weikang Shi, Bin Zou, Bowen Zheng, Haoxuan Che, Chang Chen, Yuyang He, Heyang Sun, Tianyu Huang, Chong Hou Choi, Cheng Gong, Han Shi, Haoli Bai, Xihui Liu, Hongsheng Li, Qifeng Chen, Chao Huang, Rui Liu, Chenyang Lei

llms agents diffusion multimodal benchmarks data-engineering

arXiv Machine Learning

Jul 16

Parallel gradient boosting for flexible estimation of conditional distributions

arXiv:2607. 13550v1 Announce Type: cross Abstract: Boosting is one of the most successful learning techniques for standard classification and regression tasks.

By R\'emy Chapelle (CESP, CB, EVDG), Nicolas Vayatis (CB), Bruno Falissard (CESP), Mohammed Sedki (CESP)

benchmarks

arXiv AI

Jul 16

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

arXiv:2607. 13099v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques.

By Z Sun, Q Jiang, S Sheng, L Xiang

llms efficiency benchmarks

arXiv AI

Jul 16

Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testing

arXiv:2607. 13085v1 Announce Type: cross Abstract: Recent autonomous penetration testing papers report high benchmark scores while adding multi-component security harnesses around frontier LLMs.

By Ananda Dhakal, Krish Neupane, Aarjan Chaudhary

llms agents benchmarks

arXiv AI

Jul 16

SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification

arXiv:2607. 13081v1 Announce Type: cross Abstract: We present nsfaguard, a guardrail framework for securing agentic AI systems against operational threats, such as prompt injection, sensitive information extraction, malicious code requests, dangerous tool misuse, and resource exhaustion.

By SingGuard Team

agents fine-tuning benchmarks safety

arXiv Machine Learning

Jul 16

DarwinLM: Evolutionary Structured Pruning of Large Language Models

arXiv:2502. 07780v4 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved significant success across various NLP tasks.

By Shengkun Tang, Oliver Sieberling, Eldar Kurtic, Zhiqiang Shen, Dan Alistarh

llms nlp efficiency benchmarks

arXiv AI

Jul 16

A Self-Evolving Agent for Longitudinal Personal Health Management

arXiv:2607. 13940v1 Announce Type: new Abstract: Personal health management unfolds over repeated encounters, yet most health AI systems treat each request in isolation.

By Haoran Li, Jiebi Deng, Tong Jin, Jinghong Han, Yuxin Wang, Zexin Wang, Qingyi Si, Weikang Gong, Xiahai Zhuang, Jia You, Wei Cheng, Jianfeng Feng, Hongcheng Guo

llms agents benchmarks safety

arXiv Machine Learning

Jul 16

Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations

arXiv:2607. 13399v1 Announce Type: cross Abstract: On-policy distillation (OPD) has become a key paradigm in LLM post-training, yet its training dynamics remain poorly understood.

By Rui Wang, Hongru Wang, Yi Chen, Boyang Xue, Tianqing Fang, Wenhao Yu, Kam-Fai Wong

llms efficiency benchmarks

arXiv Machine Learning

Jul 16

Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning

arXiv:2605. 07914v2 Announce Type: replace Abstract: Sharpness-aware and gradient-alignment methods have been shown to improve generalization, however each family of methods targets a single geometric property of the loss landscape, while ignoring the other.

By Aristotelis Ballas, Christos Diou

benchmarks safety

arXiv Machine Learning

Jul 16

PiVoT: A Variational Solution for Real-time Large-scale Multi-object Detection and Tracking under Heavy Clutter

arXiv:2607. 13891v1 Announce Type: new Abstract: Multi-object detection and tracking from noisy point clouds remain challenging in many data-scarce radar applications.

By Runze Gan, Qing Li, Simon J. Godsill, Mike E. Davies, James R. Hopgood

computer-vision efficiency benchmarks

arXiv Machine Learning

Jul 16

Leveraging unlabelled data for generalizable neural population decoding

arXiv:2607. 14086v1 Announce Type: new Abstract: Robust and accurate neural decoders are integral to neurotechnologies such as brain-computer interfaces and closed-loop experiments.

By Ximeng Mao, Nanda H. Krishna, Avery Hee-Woon Ryoo, Matthew G. Perich, Guillaume Lajoie

fine-tuning benchmarks

arXiv Machine Learning

Jul 16

GFlowRL: Scaling Distribution-Matching RL to Large Language Models

arXiv:2607. 13394v1 Announce Type: cross Abstract: Generative Flow Networks (GFlowNets) offer a promising alternative to reward-maximizing reinforcement learning (RL) for large reasoning models, encouraging diverse reasoning paths by matching reward distributions rather than collapsing to dominant modes.

By Xiaodong Liu, Michael Xu, Jack W. Stokes, Paul Smolensky, Doug Burger, Jianfeng Gao

llms reinforcement-learning multimodal benchmarks safety

arXiv AI

Jul 16

UTS at ELOQUENT 2026 Voight-Kampff: structural shifts in AI writing bypass state-of-the-art detectors

arXiv:2607. 13565v1 Announce Type: cross Abstract: We investigate which language model evasion attacks survive state-of-the-art adversarial fine-tuning, developing strategies that sweep the top 5 positions on the ELOQUENT 2026 Voight-Kampff leaderboard.

By Dima Galat, Marian-Andrei Rizoiu

llms fine-tuning benchmarks safety

arXiv AI

Jul 16

STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle

arXiv:2607. 13618v1 Announce Type: new Abstract: LLM agents are increasingly evaluated on multi-week decision tasks in which the state that drives cost is never directly observed.

By Sagar Deb, Ashwanth Krishnan

llms agents reinforcement-learning benchmarks

Get the daily briefing