Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,090 stories · RSS feed

arXiv AI

Jun 4

MedForge: Interpretable Medical Deepfake Detection via Forgery-aware Reasoning

arXiv:2603. 18577v2 Announce Type: replace Abstract: Text-guided image editors can now manipulate authentic medical scans with high fidelity, enabling lesion implantation/removal that threatens clinical trust and safety.

By Zhihui Chen, Kai He, Qingyuan Lei, Bin Pu, Jian Zhang, Yuling Xu, Mengling Feng

benchmarks safety

arXiv Machine Learning

Jun 4

Ternary Decision Trees with Locally-Adaptive Uncertainty Zones

arXiv:2605. 22740v2 Announce Type: replace Abstract: Decision trees assign identical confidence to instances near and far from each split threshold.

By William Smits

benchmarks

arXiv AI

Jun 4

Geometry-Aware Hallucination Detection in Large Language Models

arXiv:2601. 06196v3 Announce Type: replace-cross Abstract: Large language models (LLMs) frequently generate factually incorrect or unsupported content, commonly referred to as hallucinations.

By Bodla Krishna Vamshi, Rohan Bhatnagar, Haizhao Yang

llms rag nlp fine-tuning benchmarks safety

arXiv AI

Jun 4

Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text

arXiv:2605. 29076v2 Announce Type: replace-cross Abstract: LLMs have advanced text classification, yet existing paradigms face a trade-off: supervised (label only) fine-tuning is scalable but offers limited reasoning on complex text and lacks broader model transparency, while discrete prompt optimization offers human-readable instructions but struggles with performance and scalability.

By Tianyang Zhou, Wenbo Chen, Pierre Jinghong Liang, Leman Akoglu

llms nlp reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv Machine Learning

Jun 4

LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding

arXiv:2606. 04302v1 Announce Type: cross Abstract: Key-value (KV) caching accelerates inference of large language models (LLMs) by reusing past computations for generated tokens.

By Haocheng Xia, Mihir Pamnani, Hanxi Fang, Supawit Chockchowwat, Yongjoo Park

llms rag efficiency benchmarks

arXiv AI

Jun 4

Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamics

arXiv:2506. 06006v3 Announce Type: replace-cross Abstract: Can unified vision-language models (VLMs) perform forward dynamics prediction (FDP), i.

By Yifu Qiu, Yftah Ziser, Anna Korhonen, Shay B. Cohen, Edoardo M. Ponti

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 4

HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series

arXiv:2605. 11130v4 Announce Type: replace-cross Abstract: Critical events in multivariate time series, from turbine failures to cardiac arrhythmias, demand accurate prediction, yet labeled data is scarce because such events are rare and costly to annotate.

By Jonas Petersen, Gian-Alessandro Lombardi, Riccardo Maggioni, Camilla Mazzoleni, Federico Martelli, Philipp Petersen

llms rag benchmarks

arXiv AI

Jun 4

MIRAGE: Mobile Agents with Implicit Reasoning and Generative World Models

arXiv:2606. 04627v1 Announce Type: new Abstract: Mobile agents are increasingly expected to operate everyday applications from screenshots and language goals, where reliable control requires reasoning over screen affordances, multi-step navigation, and future state changes.

By Zhichao Yang, Yuanze Hu, Haojie Hao, Longkun Hao, Dongshuo Huang, Hongyu Lin, Gen Li, Lanqing Hong, Yihang Lou, Yan Bai

agents diffusion fine-tuning benchmarks

arXiv AI

Jun 4

LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection

arXiv:2606. 04050v1 Announce Type: cross Abstract: Existing quantization methods are fundamentally limited by rigid, integer-based bit-widths (e.

By Liulu He, XuanAng Liu, Juntao Liu, Taolue Feng, Ting Lu, Chunsheng Gan, Zhiyv Peng, Yuan Du, Huanrui Yang, Yijiang Liu, Li Du

llms efficiency benchmarks

arXiv AI

Jun 4

Aryabhata 2: Scaling Reinforcement Learning for Advanced STEM Reasoning

arXiv:2605. 28829v2 Announce Type: replace-cross Abstract: Competitive STEM examinations such as JEE and NEET require multi-step symbolic reasoning, precise numerical computation, and deep conceptual understanding across physics, chemistry, and mathematics.

By Ritvik Rastogi, Vishal Singh, Tejas Chaudhari, Sandeep Varma

llms reinforcement-learning benchmarks

arXiv AI

Jun 4

Lost in Fog: Sensor Perturbations Expose Reasoning Fragility in Driving VLAs

arXiv:2605. 21446v2 Announce Type: replace-cross Abstract: Interpretable autonomous driving planners depend not only on generating explanations, but also on those explanations remaining reliable under real-world sensor degradation.

By Abhinaw Priyadershi, Jelena Frtunikj

agents robotics multimodal benchmarks safety

arXiv AI

Jun 4

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization

arXiv:2505. 11166v3 Announce Type: replace-cross Abstract: Despite advances in pretraining with extended context sizes, large language models (LLMs) still face challenges in effectively utilizing real-world long-context information, primarily due to insufficient long-context alignment caused by data quality issues, training inefficiencies, and the lack of well-designed optimization objectives.

By Huashan Sun, Shengyi Liao, Yansen Han, Yu Bai, Yang Gao, Cheng Fu, Weizhou Shen, Fanqi Wan, Ming Yan, Ji Zhang, Fei Huang

llms benchmarks safety data-engineering

arXiv AI

Jun 4

GenSpan: Generation-Calibrated Motion Span Priors for Multi-Verb Video Corpus Moment Retrieval

arXiv:2603. 22121v2 Announce Type: replace-cross Abstract: Video Corpus Moment Retrieval (VCMR) aims to retrieve both the correct video and its temporal segment corresponding to a natural-language query, a task that is especially challenging for multi-verb queries where temporal action ordering is critical.

By Yunzhuo Sun, Xinyue Liu, Yanyang Li, Nanding Wu, Linlin Zong, Xianchao Zhang, Wenxin Liang

llms multimodal benchmarks

arXiv Machine Learning

Jun 4

EpiFormer: Learning Antigen-Antibody Interactions for Epitope Prediction via Geometric Deep Learning

arXiv:2606. 04154v1 Announce Type: cross Abstract: Antibodies neutralize foreign antigens by binding to specific surface regions called epitopes.

By Mansoor Ahmed, Huirong Chai, Haoxin Wang, Hemanth Venkateswara, Murray Patterson

efficiency benchmarks

arXiv AI

Jun 4

Learning Long Range Spatio-Temporal Representations over Continuous Time Dynamic Graphs with State Space Models

arXiv:2606. 04672v1 Announce Type: cross Abstract: Continuous-time dynamic graphs (CTDGs) provide a richer framework to capture fine-grained temporal patterns in evolving relational data.

By Ayushman Raghuvanshi, Thummaluru Siddartha Readdy, Sundeep Prabhakar Chepuri, Mahesh Chandran

benchmarks

arXiv Machine Learning

Jun 4

veriFIRE: an Industrial Case Study in Verifying Consistency Properties for a DNN-Based Wildfire Detection System

arXiv:2606. 04121v1 Announce Type: cross Abstract: We present our ongoing work on the veriFIRE project: a collaboration between industry and academia, aimed at applying verification to increase the reliability of a real-world, safety-critical system.

By Idan Refaeli, Maya Swisa, Itay Buchnik, Alon Zada, Guy Amir, Elad Mandelbaum, Ziv Freund, Guy Katz

benchmarks safety

arXiv AI

Jun 4

Caught in the Act(ivation): Toward Pre-Output and Multi-Turn Detection of Credential Exfiltration by LLM Agents

arXiv:2606. 04141v1 Announce Type: cross Abstract: LLM agents often place sensitive credentials in the same context window as untrusted retrieved content, creating a direct path for indirect prompt injection to induce credential exfiltration.

By Kargi Chauhan, Pratibha Revankar

llms agents benchmarks

arXiv AI

Jun 4

Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

arXiv:2606. 04238v1 Announce Type: cross Abstract: Aggressive weight quantization to 2-bit precision offers substantial throughput and memory gains for large language model (LLM) inference, but typically incurs severe accuracy degradation.

By Devleena Das, Rajeev Patwari, Elliott Delaye, Ashish Sirasao

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 4

PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?

arXiv:2602. 01146v2 Announce Type: replace Abstract: Conversational assistants are increasingly integrating long-term memory with large language models (LLMs).

By Sidharth Pulipaka, Oliver Chen, Manas Sharma, Taaha S Bajwa, Vyas Raina, Ivaxi Sheth

llms benchmarks safety

arXiv AI

Jun 4

Luminol-AIDetect: Fast Zero-shot Machine-Generated Text Detection based on Perplexity under Text Shuffling

arXiv:2604. 25860v2 Announce Type: replace-cross Abstract: Machine-generated text (MGT) detection requires identifying structurally invariant signals across generation models, rather than relying on model-specific fingerprints.

By Lucio La Cava, Andrea Tagarelli

llms benchmarks safety

Get the daily briefing