Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,065 stories · RSS feed

arXiv AI

Jul 22

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

arXiv:2607. 19345v1 Announce Type: cross Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier.

By Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

llms reinforcement-learning benchmarks

arXiv AI

Jul 22

Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing

arXiv:2607. 19064v1 Announce Type: cross Abstract: Large-scale visual generators are increasingly capable but costly to train, fine-tune, and deploy.

By Xinjie Zhang, Peng Zhang, Shicheng Zheng, Jinghao Guo, Zhaoyang Jia, Yifei Shen, Xun Guo, Yuxuan Luo, Jiahao Li, Wenxuan Xie, Fanyi Pu, Xiaoyi Zhang, Kaichen Zhang, Zongyu Guo, Tianci Bi, Dongnan Gui, Zhening Liu, Zimo Wen, Zihan Zheng, Senqiao Yang, Xiao Li, Jinglu Wang, Bin Li, Yan Lu

llms diffusion nlp reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 22

OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining

arXiv:2607. 18927v1 Announce Type: new Abstract: We present OntoBook, a method that converts medical ontology structure into pretraining signal for encoder language models.

By Rian Touchent (ALMAnaCH), \'Eric de la Clergerie (ALMAnaCH)

llms benchmarks safety

arXiv AI

Jul 22

Physics-Guided Masked Multi-Task Network for Edge-Friendly Battery Health Diagnostics from Sto-chastically Fragmented Charging Profiles

arXiv:2607. 18330v1 Announce Type: cross Abstract: The deployment of reliable lithium-ion battery management systems is crucial for accelerating electrification, yet the joint prognosis of State of Health (SOH) and Remaining Useful Life (RUL) remains severely hindered by task heteroscedasticity.

By Shuhao Chen, Tianyu Shi, Chengyi Tu

llms rag benchmarks

arXiv AI

Jul 22

Dynamic Loss Balancing for Joint SOH and RUL Prediction of Lithium-Ion Batteries via a Rotary SOH-Injected Prior Battery Transformer

arXiv:2607. 18329v1 Announce Type: cross Abstract: The deployment of reliable lithium-ion battery management systems is crucial for accelerating electrification, yet the joint prognosis of State of Health (SOH) and Remaining Useful Life (RUL) remains severely hindered by task heteroscedasticity.

By Shuhao Chen, Tianyu Shi, Yiwen Huang, Chengyi Tu

llms rag benchmarks

arXiv Machine Learning

Jul 22

One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models

arXiv:2607. 19128v1 Announce Type: new Abstract: Vision-language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general-purpose backbones for graph-structured data remains largely unexplored.

By Jiayi Yang, Yifang Chen, Yuanfu Sun, Jiajin Liu, Qiaoyu Tan

llms rag multimodal benchmarks

arXiv AI

Jul 22

Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression

arXiv:2607. 18284v1 Announce Type: cross Abstract: To excel at their domain large language models are comprised of billions of parameters.

By Athanasios Ntovas, Alexandros Doumanoglou, Petros Drakoulis, Dimitris Zarpalas

llms efficiency benchmarks

arXiv Machine Learning

Jul 22

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

arXiv:2607. 19223v1 Announce Type: new Abstract: Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference.

By Yu-Yang Qian, Hao-Cong Wu, Chen Chen, Jiacheng Sun, Zhenhua Dong, Peng Zhao, Zhi-Hua Zhou

llms diffusion efficiency benchmarks

arXiv AI

Jul 22

Operational Hallucination and Safety Drift in AI Agents

arXiv:2607. 18366v1 Announce Type: new Abstract: Large language models (LLMs) serving as planners in tool-using autonomous agents introduce dynamic reliability risks in multi-turn execution.

By Shasha Yu, Fiona Carroll, Barry L. Bentley

llms agents benchmarks safety

arXiv AI

Jul 22

Benchmarking Generalization in Financial Statement Fraud Detection: robust evaluation and novel tasks

arXiv:2607. 19259v1 Announce Type: cross Abstract: Financial statement fraud detection (FSFD) is crucial for market integrity but faces challenges from increasingly sophisticated schemes and under-utilized textual data in financial reports.

By Guy Stephane Waffo Dzuyo (Forvis Mazars, LORIA CNRS Universit\'e de Lorraine), Ga\"el Guibon (LORIA CNRS Universit\'e de Lorraine, LIPN CNRS Universit\'e Sorbonne Paris Nord), Christophe Cerisara (LORIA CNRS Universit\'e de Lorraine), Luis Belmar-Letelier (Forvis Mazars)

llms benchmarks

arXiv Machine Learning

Jul 22

Breaking Feedback-Blindness: Utility-Augmented Transformer for Sequential Decision Making

arXiv:2607. 18910v1 Announce Type: new Abstract: Sequential decision making in non-stationary and partially observable environments requires rapid adaptation to latent regime changes.

By Yuyang Shen, Shan Dai, Daimin Chen

llms benchmarks

arXiv AI

Jul 22

SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurement

arXiv:2607. 18960v1 Announce Type: cross Abstract: Procuring supervised fine-tuning (SFT) data forces a buyer to decide, before any downstream training, whether a candidate corpus is worth acquiring.

By Arther Tian, Alex Ding, Simon Wu, Aaron Chan

llms fine-tuning benchmarks

arXiv AI

Jul 22

Data Leakage Prevention in Agentic Applications via Preemptive Hardening

arXiv:2607. 18847v1 Announce Type: cross Abstract: Agentic systems integrate LLM driven planning with interfaces to external tools, making data leakage and tool misuse feasible via instruction/data boundary failures and prompt injection attacks.

By Akansha Shukla, Emily Bellov, Parth Atulbhai Gandhi, Yuval Elovici, Asaf Shabtai

llms agents robotics benchmarks safety

arXiv AI

Jul 22

Decoupled Pipeline with Proposal Reranking and Score Fusion for Positive-Unlabeled Marine Species Detection

arXiv:2607. 18700v1 Announce Type: cross Abstract: The FathomNetCLEF 2026 competition combines underwater object detection and fine-grained marine species classification under a positive-unlabeled evaluation setting.

By Robert James Brock, Sebastian Maximilian Krupa, Jason Kahei Tam

rag computer-vision fine-tuning benchmarks

arXiv Machine Learning

Jul 22

Subject-Conditioned Glucose Forecasting in Type-1 Diabetes

arXiv:2607. 19006v1 Announce Type: new Abstract: Accurate forecasting of blood glucose concentration is key in the management of Type 1 Diabetes, facilitating early detection of adverse glycemic events and supporting timely therapeutic interventions.

By Giorgia Rigamonti, Mirko Paolo Barbato, Davide Marelli, Paolo Napoletano

multimodal benchmarks

arXiv AI

Jul 22

AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents

arXiv:2607. 18754v1 Announce Type: new Abstract: LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.

By Kunlun Zhu, Xuyan Ye, Zhiguang Han, Yuchen Zhao, Bingxuan Li, Weijia Zhang, Muxin Tian, Xiangru Tang, Pan Lu, James Zou, Jiaxuan You, Heng Ji

llms agents benchmarks

arXiv AI

Jul 22

BearingNAS: Obtaining In-Sensor Intelligent Fault Diagnosis Systems for Bearings Using a Laptop

arXiv:2607. 18287v1 Announce Type: cross Abstract: This paper introduces BearingNAS, a Hardware-Aware Neural Architecture Search (HW-NAS) framework designed to shift the intelligence directly onto the sensor die via in-sensor processing.

By Andrea Mattia Garavagno, Edoardo Ragusa, Paolo Gastaldo, Antonio Frisoli, Rodolfo Zunino

benchmarks

arXiv AI

Jul 22

Supra Cognitive Modes: A Routed Architecture for Agent Memory

arXiv:2607. 19096v1 Announce Type: new Abstract: Agent-memory workloads mix direct factual lookup, relation-chain and current-state reasoning, and broad synthesis over long histories.

By Joshua Tobkin, David Yang

rag agents benchmarks safety

arXiv AI

Jul 22

What the Waveform Knows: Transparent-first Speech and Audio Intelligence with Caption Studio

arXiv:2607. 18704v1 Announce Type: cross Abstract: Caption Studio is a transparency-first speech and audio intelligence platform that transforms spoken audio and video into structured, searchable content through automated transcription, speaker diarization, speech analytics, signal-level audio analysis, and subtitle generation.

By Cheng Siong Chin, Jianhua Zhang, Mohan Venkateshkumar

multimodal benchmarks safety

arXiv AI

Jul 22

Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agents

arXiv:2607. 18826v1 Announce Type: cross Abstract: LLM-agent defenses are typically evaluated one session at a time.

By SangJin Park, Myungsub Choi, Jineok Kim, Minseung Kang

llms agents benchmarks safety

Get the daily briefing