Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,285 stories · RSS feed

arXiv AI

Jul 2

Learning User-Aware Recall: Personalized Retrieval in Long-Term Conversational Memory

arXiv:2607. 00017v1 Announce Type: cross Abstract: Long-term conversational agents are expected to remember past interactions, but memory is useful only when the right evidence is recalled for the right user.

By ZhiShu Jiang, Haibo Liu, Xin Shen, Guanqiang QI, Chenxi Miao, Weikang Li, Liwei Qian, Xin Pei, Jizhou Huang

llms agents benchmarks

arXiv Machine Learning

Jul 2

StateFlow: Dual-State Recurrent Modeling for Long-Horizon Time Series Forecasting

arXiv:2607. 00197v1 Announce Type: new Abstract: Long-horizon multivariate time series forecasting (LTSF) remains challenging due to non-stationarity, regime shifts, and error accumulation.

By Haroon Gharwi, Yue Dai, Kai Shu

llms benchmarks

arXiv AI

Jul 2

Continuous Knowledge Metabolism: Generating Scientific Hypotheses from Evolving Literature

arXiv:2604. 12243v2 Announce Type: replace-cross Abstract: Identifying promising research directions in fast-moving subareas is one of the most cognitively expensive tasks in modern AI research.

By Jinkai Tao, Yubo Wang, Xiaoyu Liu, Menglin Yang

llms agents benchmarks

arXiv Machine Learning

Jul 2

Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalization

arXiv:2607. 00796v1 Announce Type: new Abstract: Visual Reinforcement Learning (VRL) has achieved considerable success in solving control tasks.

By Jinwen Wang, Youfang Lin, Xiaobo Hu, Qian Xu, Shuo Wang, Zhuo Chen, Kai Lv

agents reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jul 2

FeLoG: Scalable and Efficient Distributed Graph Embedding with Feedback Loop Mechanism

arXiv:2606. 22180v2 Announce Type: replace-cross Abstract: Graph embedding maps graph nodes into low-dimensional vectors to support applications such as recommendation, fraud detection, and graph-based retrieval-augmented generation (GraphRAG).

By Peng Fang, Arijit Khan, Ziqiang Wu, Zhenli Li, Yibo Zhou, Fang Wang, Dan Feng

rag benchmarks

arXiv AI

Jul 2

SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing

arXiv:2607. 00208v1 Announce Type: cross Abstract: Reinforcement learning for diffusion large language models (dLLMs) has largely moved to trajectory-aware methods.

By Ruikang Zhao, Zhenting Wang, Han Gao, Ligong Han

llms diffusion reinforcement-learning benchmarks

arXiv AI

Jul 2

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale

arXiv:2602. 05711v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) architectures are evolving towards finer granularity to improve parameter efficiency.

By Jingze Shi, Zhangyang Peng, Yizhang Zhu, Yifan Wu, Guang Liu, Yuyu Luo

efficiency benchmarks

arXiv Machine Learning

Jul 2

Efficient Neural Controlled Differential Equations via Attentive Kernel Smoothing

arXiv:2602. 02157v2 Announce Type: replace Abstract: Neural Controlled Differential Equations (Neural CDEs) provide a powerful continuous-time framework for sequence modeling, yet the roughness of the driving control path often restricts their efficiency.

By Egor Serov, Ilya Kuleshov, Alexey Zaytsev

benchmarks

arXiv Machine Learning

Jul 2

GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems

arXiv:2607. 01128v1 Announce Type: new Abstract: Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation.

By Meenakshi Krishnan, Pranav Pulijala, Ke Chen, Haizhao Yang, Ramani Duraiswami

llms benchmarks

arXiv Machine Learning

Jul 2

A Lightweight Self-Supervised Learning Framework for Multivariate Time Series using Hierarchical-JEPA on ECG Data

arXiv:2607. 01145v1 Announce Type: new Abstract: Data analysis in the medical domain often encounters scenarios involving a limited target dataset and a large, unannotated dataset with a general distribution.

By Siwon Kim

llms rag computer-vision benchmarks

arXiv Machine Learning

Jul 2

Learning dynamical systems from noisy data with Weak-form Kernel Ridge Regression

arXiv:2607. 00257v1 Announce Type: new Abstract: Accurate prediction of complex dynamical systems from noisy measurements remains a significant challenge in scientific computing.

By Max Kreider, John Harlim, Daning Huang

benchmarks safety

arXiv AI

Jul 2

KGS-GCN: Kinematics-Driven Gaussian Splatting and Probabilistic Topology for Skeleton-Based Action Recognition

arXiv:2603. 16943v2 Announce Type: replace-cross Abstract: Skeleton-based action recognition is widely applied in sensor-based systems, including human-computer interaction and intelligent surveillance.

By Yuhan Chen, Yicui Shi, Guofa Li, Liping Zhang, Jie Li, Jiaxin Gao, Wenbo Chu

benchmarks

arXiv AI

Jul 2

MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

arXiv:2607. 01071v1 Announce Type: cross Abstract: Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators.

By Zhishang Xiang, Zerui Chen, Yunbo Tang, Zhimin Wei, Ruqin Ning, Yujie Lin, Qinggang Zhang, Jinsong Su

llms agents benchmarks

arXiv AI

Jul 2

EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards

arXiv:2607. 00218v1 Announce Type: cross Abstract: Vision-language models (VLMs) are now proposed as runtime safety guards for embodied agents in homes and factories.

By Siddhant Panpatil, Arth Singh, Mijin Koo, Chaeyun Kim, Haon Park, Dasol Choi

llms agents robotics multimodal benchmarks safety

arXiv AI

Jul 2

Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences

arXiv:2607. 00832v1 Announce Type: cross Abstract: A single panorama captures the full visual sphere from one camera center, yet confines users to looking around in place without enabling true scene exploration.

By Zhenjia Li, Jinrang Jia, Yifeng Shi

diffusion benchmarks

arXiv AI

Jul 2

Learning to Compose: Revisiting Proxy Task Design for Zero-Shot Composed Image Retrieval

arXiv:2607. 00374v1 Announce Type: cross Abstract: Composed Image Retrieval (CIR) retrieves a target image from a reference image and a textual modification.

By Jingjing Zhang, Lei Zhang, Zheren Fu, Zhendong Mao

multimodal benchmarks

arXiv AI

Jul 2

Entropy-Regularized Probabilistic Gates for Sparse Model Discovery in Scarce-Data Federated Learning

arXiv:2607. 00275v1 Announce Type: cross Abstract: Federated Learning (FL) is a distributed machine learning (ML) paradigm with collaboration among multiple clients without sharing data.

By Krishna Harsha Kovelakuntla Huthasana, Alireza Olama, Andreas Lundell

efficiency benchmarks

arXiv AI

Jul 2

Hate Speech Detection in Turkish and Arabic Languages: A Comprehensive Study

arXiv:2607. 00143v1 Announce Type: cross Abstract: Online hate speech has been linked to a global rise in violence against minorities, including incidents such as mass shootings, lynchings, and ethnic cleansing.

By Somaiyeh Dehghan, G\"ok\c{c}e Uludo\u{g}an, Mehmet Umut \c{S}en, Elif Erol, Arzucan \"Ozg\"ur, Berrin Yanikoglu

benchmarks

arXiv AI

Jul 2

An LLM-Based Framework for Intent-Driven Network Topology Design

arXiv:2607. 00292v1 Announce Type: cross Abstract: Designing deployable and resilient network topologies from natural language requirements remains a challenging problem in network automation.

By Kholoud El-Habbouli, Fen Zhou, Stephane Huet

llms benchmarks

arXiv AI

Jul 2

DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

arXiv:2606. 31980v1 Announce Type: cross Abstract: Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves?

By Meng Chen, Anya Ji, Tsung-Han Wu, Tobias Maringgele, David M. Chan, Alane Suhr, Amy Pavel

agents multimodal benchmarks

Get the daily briefing