Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,880 stories · RSS feed

arXiv Machine Learning

Jun 11

UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning

arXiv:2606. 12372v1 Announce Type: cross Abstract: Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance.

By Haoyuan Deng, Yitong Gao, Yudong Lin, Haichao Liu, Zhenyu Wu, Ziwei Wang

agents reinforcement-learning robotics benchmarks

arXiv AI

Jun 11

Grounding Computer Use Agents on Human Demonstrations

arXiv:2511. 07332v2 Announce Type: replace-cross Abstract: Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen elements.

By Aarash Feizi, Shravan Nayak, Xiangru Jian, Kevin Qinghong Lin, Kaixin Li, Rabiul Awal, Xing Han L\`u, Johan Obando-Ceron, Juan A. Rodriguez, Nicolas Chapados, David Vazquez, Adriana Romero-Soriano, Reihaneh Rabbany, Perouz Taslakian, Christopher Pal, Spandana Gella, Sai Rajeswar

agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 11

CoVar: Confidence-Variance-Guided Pseudo-Label Selection for Semi-Supervised Learning

arXiv:2601. 11670v3 Announce Type: replace-cross Abstract: Pseudo-label selection in semi-supervised learning is commonly driven by maximum-confidence thresholds, yet confidence alone can be unreliable under model overconfidence and class imbalance.

By Jinshi Liu, Lei He, Pan Liu

computer-vision benchmarks

arXiv AI

Jun 11

Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning

arXiv:2601. 00791v2 Announce Type: replace-cross Abstract: Verifying whether a language model is genuinely reasoning or pattern-matching remains an open problem: learned verifiers are expensive, and output-based heuristics are brittle.

By Valentin No\"el

llms rag benchmarks

arXiv Machine Learning

Jun 11

Analytic Bijections for Smooth and Interpretable Normalizing Flows

arXiv:2601. 10774v2 Announce Type: replace Abstract: A key challenge in normalizing flows is finding expressive invertible scalar bijections.

By Mathis Gerdes, Miranda C. N. Cheng

benchmarks

arXiv Machine Learning

Jun 11

Context-Driven Incremental Compression for Multi-Turn Dialogue Generation

arXiv:2606. 12411v1 Announce Type: cross Abstract: Modern conversational agents condition on an ever-growing dialogue history at each turn, incurring redundant attention and encoding costs that grow with conversation length.

By Yeongseo Jung, Jaehyeok Kim, Eunseo Jung, Jiachuan Wang, Yongqi Zhang, Ka Chun Cheung, Simon See, Lei Chen

agents nlp efficiency benchmarks

arXiv Machine Learning

Jun 11

Steering the Noise: Turning Random Perturbations into Effective Descent for Memory-Efficient LLM Fine-Tuning

arXiv:2601. 04710v2 Announce Type: replace-cross Abstract: Fine-tuning large language models (LLMs) achieves strong performance but is often limited by the memory overhead of backpropagation.

By Feihu Jin, Shipeng Cen, Ying Tan

llms fine-tuning benchmarks

arXiv AI

Jun 11

Agentic Software: How AI Agents Are Restructuring the Software Paradigm

arXiv:2606. 05608v2 Announce Type: replace-cross Abstract: For over half a century, software engineering has operated on a foundational premise: human engineers decompose problems, encode decision logic into static code, and manually adapt that code as requirements evolve.

By Zhenfeng Cao

llms agents benchmarks

arXiv AI

Jun 11

Automated Creativity Evaluation of Language Models Across Open-Ended Tasks

arXiv:2606. 11762v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential.

By Min Sen Tan, Zachary Kit Chun Choy, Syed Ali Redha Alsagoff, Nadya Yuki Wangsajaya, Mohor Banerjee, Swaagat Bikash Saikia, Alvin Chan

llms rag agents benchmarks

arXiv AI

Jun 11

When Poison Fails After Retrieval: Revisiting Corpus Poisoning under Chunking and Reranking Pipelines

arXiv:2606. 11265v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems are vulnerable to corpus poisoning attacks that manipulate downstream model outputs through malicious knowledge injection.

By Xi Nie, Hongwei Li, Shenghao Wu, Mingxuan Li, Jiachen Li, Wenbo Jiang

rag benchmarks safety

arXiv AI

Jun 11

Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction

arXiv:2606. 11909v1 Announce Type: new Abstract: Benchmarks are essential for evaluating embodied spatial intelligence, yet their construction is labor-intensive, hard to reuse, and difficult to maintain.

By Baoyang Jiang, Fengchun Zhang, Leyuan Wang, Haotian Li, Yida Wang, Zhe Ji, Jinshan Lai, Xi Ren, Jianwei Hu, Qiang Ma

agents robotics benchmarks

arXiv AI

Jun 11

Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies

arXiv:2602. 18291v2 Announce Type: replace Abstract: Online Multi-Agent Reinforcement Learning (MARL) is a prominent framework for efficient agent coordination.

By Zhuoran Li, Hai Zhong, Xun Wang, Qingxin Xia, Lihua Zhang, Longbo Huang

agents diffusion reinforcement-learning multimodal benchmarks

arXiv AI

Jun 11

KAN-MLP-Mixer: A comprehensive investigation of the usage of Kolmogorov-Arnold Networks (KANs) for improving IMU-based Human Activity Recognition

arXiv:2605. 19031v2 Announce Type: replace Abstract: Kolmogorov-Arnold Networks (KANs) have demonstrated an exceptional ability to learn complex functions on clean, low-dimensional data but struggle to maintain performance on noisy and imperfect real-world datasets.

By Mengxi Liu, Sizhen Bian, Vitor Fortes, Francisco Calatrava Nicolas, Daniel Gei{\ss}ler, Maximilian Kiefer-Emmanouilidis, Bo Zhou, Paul Lukowicz

rag benchmarks

arXiv Machine Learning

Jun 11

LatticeBridge: Rare-Event Sequential Inference for Faithful Structured Sequence Synthesis

arXiv:2606. 11203v1 Announce Type: cross Abstract: Structured sequence generation often requires a model to satisfy several input-derived constraints in a single output.

By Faruk Alpay, Bugra Kilictas

llms benchmarks

arXiv Machine Learning

Jun 11

GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMs

arXiv:2606. 11898v1 Announce Type: cross Abstract: Research on Text-Attributed Graphs (TAGs) has gained significant attention recently due to its broad applications across various real-world data scenarios, such as citation networks, e-commerce platforms, social media, and web pages.

By Hengyi Feng, Zeang Sheng, Meiyi Qiang, Meiyi Qiang, Wentao Zhang

llms rag benchmarks safety

arXiv Machine Learning

Jun 11

OGPO: Sample Efficient Full-Finetuning of Generative Control Policies

arXiv:2605. 03065v2 Announce Type: replace Abstract: Generative control policies (GCPs), such as diffusion- and flow-based control policies, have emerged as effective parameterizations for robot learning.

By Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Dai, Paarth Shah, Max Simchowitz

diffusion reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jun 11

Are LLMs Bad at Moral Reasoning?

arXiv:2606. 11635v1 Announce Type: cross Abstract: For highly capable AI systems to operate safely in dynamic, open-ended environments, they must be able to identify, understand, and respond to moral reasons for action, and constrain their behaviour accordingly.

By Menghang Zhu, Seth Lazar

llms benchmarks

arXiv Machine Learning

Jun 11

FOCUS: DLLMs Know How to Tame Their Compute Bound

arXiv:2601. 23278v2 Announce Type: replace Abstract: Diffusion Large Language Models (DLLMs) offer a compelling alternative to Auto-Regressive models, but their deployment is constrained by high decoding cost.

By Kaihua Liang, Xin Tan, An Zhong, Hong Xu, Marco Canini

llms diffusion efficiency benchmarks

arXiv Machine Learning

Jun 11

Modelling magnetic material properties with uncertainty-aware neural networks

arXiv:2606. 11870v1 Announce Type: cross Abstract: Machine learning is increasingly applied to accelerate the discovery of novel materials by exploring large compositional and structural design spaces.

By Clemens Wager, Heisam Moustafa, Alexander Kovacs, Qais Ali, Harald Oezelt, Hayate Yamano, Masao Yano, Noritsugu Sakuma, Hyuga Hosoi, Akihito Kinoshita, Tetsuya Shoji, Akira Kato, Thomas Schrefl

benchmarks

Hugging Face Trending Papers

Jun 11

Interpretable Factor Decomposition for Decision Intelligence in Large-Scale Financial Markets: Evidence from China's A-Share Market

We present an interpretable machine learning pipeline to decompose Cross-Sectional Equity Return Predictability into auditable factor contribution. We apply an XGBoost model with TreeSHAP attribution and conduct stress testing on 3632 Chinese A-share stocks from 2009 until 2019.

benchmarks

Get the daily briefing