Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

17,521 stories · RSS feed

arXiv AI
Jul 9

NonTextual Target Attack

arXiv:2510. 02999v5 Announce Type: replace-cross Abstract: Existing gradient-based jailbreak attacks on Large Language Models (LLMs) typically optimize adversarial suffixes to align the LLM output with predefined target responses.

By Xinzhe Huang, Wenjing Hu, Tianhang Zheng, Kedong Xiu, Hongsheng Hu, Xiaojun Jia, Di Wang, Zhan Qin, Kui Ren
arXiv Machine Learning
Jul 9

Creating Power Distribution Network Layouts Using Generative Adversarial Networks and Image-Based Representations

arXiv:2607. 06622v1 Announce Type: cross Abstract: Utilities increasingly rely on planning and operational tools to cope with the increased penetrations of distributed energy resources, yet the lack of realistic, openly available datasets remains a major barrier for benchmarking and comparison.

By Juan Manuel Garcia-Perez, Carlos Mateo
arXiv Machine Learning
Jul 9

Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector

arXiv:2607. 07382v1 Announce Type: new Abstract: Fast Radio Bursts (FRBs) are millisecond-duration radio transients whose automated detection increasingly relies on highly specialized deep learning models.

By Raiff H. Santos, Amilcar R. Queiroz, Tharcisyo S. S. Duarte, K. E. L. de Farias, Rafael A. Batista
arXiv AI
Jul 9

When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs

arXiv:2607. 07395v1 Announce Type: cross Abstract: Reliable confidence estimation remains a key limitation of test-time adaptation in vision-language models (VLMs), where prompt tuning improves zero-shot accuracy but often degrades calibration due to entropy-driven overconfidence.

By Tanay Sodha, Aditya Sharma, Ramya Hebbalaguppe, Vinti Agarwal, Pranav Murthy Yeluripaty
arXiv Machine Learning
Jul 9

Robust Federated Learning Under Real-World Client Churn

arXiv:2607. 06979v1 Announce Type: new Abstract: Federated Learning (FL) enables training shared models on private, on-device data, but production deployments remain constrained to slow, multi-day refresh cycles due to the complexity of coordinating massive client populations.

By Dhruv Garg, Neha Lakhani, Debopam Sanyal, Myungjin Lee, Alexey Tumanov, Ada Gavrilovska