Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,077 stories · RSS feed

arXiv AI

Jul 20

MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

arXiv:2607. 15592v1 Announce Type: new Abstract: Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues.

By Xu Hou, Meiyu Liang, Wei Huang, Yawen Li, Zhe Xue, Wu Liu, Guanhua Ye, Lei Shi, Kangkang Lu

llms diffusion multimodal benchmarks safety

arXiv AI

Jul 20

Loop the Loopies!

arXiv:2607. 16051v1 Announce Type: cross Abstract: We present Loopie, the most powerful looped Transformer to date.

By Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang, Ran Tao, Joey Zhou, Bryan Dai

llms benchmarks

arXiv AI

Jul 20

Field-Aware RankMixer with Dual-Stream Bilinear Fusion for the Tencent UNI-REC Challenge

arXiv:2607. 15590v1 Announce Type: cross Abstract: This paper presents our solution to the KDD Cup 2026 Tencent UNIREC Challenge.

By Yufeng Zhang, Zhengqi Xu, Jiajun Cui

benchmarks

arXiv AI

Jul 20

Cura 1T: Specialized Model for Agentic Healthcare

arXiv:2607. 15314v1 Announce Type: new Abstract: Healthcare spans high-stakes communication, expert reasoning, and workflow execution, yet specialized LLMs that cover these use cases together remain limited.

By actAVA AI, :, Haolin Chen, Leon Qi, Steve Brown, Deon Metelski, Tao Xia, Joonyul Lee, Qixuan Wang, Kevin Riley, Frank Wang, Weiran Yao

llms agents benchmarks

arXiv Machine Learning

Jul 20

Multi-marginal temporal Schr\"odinger Bridge Matching from unpaired data

arXiv:2510. 01894v3 Announce Type: replace Abstract: Many natural dynamic processes -- such as in vivo cellular differentiation or disease progression -- can only be observed through the lens of static sample snapshots.

By Thomas Gravier, Thomas Boyer, Auguste Genovesio

diffusion benchmarks

arXiv AI

Jul 20

CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data

arXiv:2607. 16122v1 Announce Type: new Abstract: Evaluations should do more than measure a models current performance.

By Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru, MohammadHossein Rezaei, Aakash Sabharwal, Yunzhong He

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 20

RobustSpeechFlow: Learning Robust Text-to-Speech Trajectories via Augmentation-based Contrastive Flow Matching

arXiv:2605. 22083v2 Announce Type: replace-cross Abstract: While flow-matching text-to-speech (TTS) achieves strong zero-shot speaker similarity and naturalness, it remains susceptible to content fidelity issues, particularly skip and repeat errors from imperfect alignment.

By Jinhyeok Yang, Hyeongju Kim, Yechan Yu, Joon Byun, Frederik Bous, Juheon Lee

diffusion multimodal benchmarks safety

arXiv AI

Jul 20

DECODEM: Data Extraction from Corporate Organizational Documents via Enhanced Methods

arXiv:2607. 15879v1 Announce Type: cross Abstract: Much empirical legal research depends on translating unstructured text into structured variables.

By Jens Frankenreiter

llms benchmarks

arXiv AI

Jul 20

Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation

arXiv:2607. 15434v1 Announce Type: cross Abstract: Multi-agent systems routinely place one AI agent in authority over another.

By Jasmine Brazilek, Maheep Chaudhary, Zoe Lu, Miles Tidmarsh

llms agents benchmarks

arXiv Machine Learning

Jul 20

Rethinking the Global Knowledge of CLIP in Training-Free Open-Vocabulary Semantic Segmentation

arXiv:2502. 06818v4 Announce Type: replace Abstract: Recent works modify CLIP to perform open-vocabulary semantic segmentation in a training-free manner (TF-OVSS).

By Jingyun Wang, Cilin Yan, Guoliang Kang

rag computer-vision benchmarks

arXiv AI

Jul 20

IMBench: A Benchmark for Intuitive Robotic Manipulation

arXiv:2607. 15641v1 Announce Type: cross Abstract: Humans combine reasoning and motor control to solve complex manipulation tasks under diverse constraints.

By Anurag Maurya, Sukhvansh Jain, Prajwal Avhad, Gautham Balachandran, Ziyi Zhou, Atharva Kshirsagar, Satyam Singh, Bowen Li. Rishabh Mukund, Ritul Singh, Jatin Vira, Suvonil Chatterjee, Devesh K. Jha

llms agents robotics multimodal benchmarks

arXiv AI

Jul 20

ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learning

arXiv:2607. 15660v1 Announce Type: new Abstract: While LLM agents demonstrate strong reasoning abilities in compact and well-defined scenarios, they struggle to maintain robustness and effectiveness when faced with large-scale, diverse, and dynamic real-world environments that demand seamless tool integration.

By Shuaiyu Zhou, Fengpeng Yue, Zengjie Hu, Yuanzhe Shen, Chenyang Zhang, feng hong, Cao Liu, Ke Zeng

llms agents reinforcement-learning benchmarks

arXiv AI

Jul 20

AgentFAIR: A Multi-Agent Collaborative Framework for FAIRness Evaluation of Geospatial Datasets

arXiv:2607. 15781v1 Announce Type: new Abstract: Geospatial datasets support applications from urban planning to climate modeling, yet consistent assessment of FAIR compliance is difficult.

By Ming Chen, Pranav Pai

llms agents benchmarks safety

arXiv Machine Learning

Jul 20

SC-JEPA: Stabilizing Latent Predictive Learning for Time-Series Anomaly Prediction

arXiv:2602. 04643v2 Announce Type: replace Abstract: Time-series anomaly prediction aims to forecast future system failures before they fully emerge, making latent predictive models such as JEPA a promising framework for capturing precursor dynamics.

By Yanan He, Yunshi Wen, Xin Wang, Tengfei Ma

diffusion efficiency benchmarks

arXiv Machine Learning

Jul 20

(MPO)$^2$: Multivariate Polynomial Optimization based on Matrix Product Operators

arXiv:2607. 15916v1 Announce Type: new Abstract: Central to machine learning and signal processing is the ability to perform universal function approximation and learn complex input-output relationships from limited numbers of observations.

By Niccol\`o Ciolli, Anders Vestergaard N{\o}rskov, Michael Kastoryano, Petr Taborsky, Morten M{\o}rup

rag benchmarks

arXiv AI

Jul 20

Kolmogorov--Arnold Networks for Small Language Models

arXiv:2607. 15525v1 Announce Type: cross Abstract: Kolmogorov--Arnold Networks (KANs) replace fixed node activations with learned one-dimensional edge functions, offering an explicit interface for interpretation and a possible alternative to transformer feed-forward networks.

By Felippe Alves, Renato Vicente

llms efficiency benchmarks

arXiv Machine Learning

Jul 20

Improving Backward Conformal Prediction via Non-Conformity Score Transformation

arXiv:2602. 01733v3 Announce Type: replace-cross Abstract: Conformal Prediction (CP) provides a statistical framework for uncertainty quantification that constructs prediction sets with coverage guarantees.

By Junxian Liu, Hao Zeng, Hongxin Wei

benchmarks

arXiv AI

Jul 20

NexForge: Scaling Agent Capabilities through Requirement-Driven Task Synthesis for LLMs

arXiv:2607. 14186v2 Announce Type: replace-cross Abstract: Synthesizing training data to scale agent capabilities in LLM post-training is bottlenecked by substrate-bound task synthesis: tasks are generated from fixed tools, repositories, or skill graphs, so expanding coverage requires manual substrate engineering, transferring to a new domain demands bespoke infrastructure, and the resulting distributions inherit substrate biases rather than reflecting real-world demand.

By Jiarong Zhao, Zhikai Lei, Zhiheng Xi, Rui Zheng, Hang Yan, Jie Zhou, Qin Chen, Liang He

llms agents fine-tuning benchmarks

arXiv AI

Jul 20

Spatial Normalization for Cross-Domain Retinal Layer Segmentation in Optical Coherence Tomography

arXiv:2607. 16065v1 Announce Type: cross Abstract: Retinal layer segmentation in Optical Coherence Tomography (OCT) is a fundamental step for extracting quantitative biomarkers of retinal structure.

By Iker Moran-Cavero, Monica Hernandez, Elvira Mayordomo, Naiara Artiaga, Beatriz Pardi\~nas, Beatriz Cordon, Elena Garcia-Martin

computer-vision benchmarks

arXiv Machine Learning

Jul 20

ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language

arXiv:2607. 15607v1 Announce Type: new Abstract: Hallucinations and artificial text in LLM-generated outputs often appear as distributional deviations between prompt and response hidden-state distributions.

By Sergey Zakharov, Rodion Oblovatny, Alexey Zaytsev

llms benchmarks safety

Get the daily briefing