Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv AI

Jul 14

Agentic Skill Optimization over Lie Algebroids

arXiv:2607. 11493v1 Announce Type: cross Abstract: Agentic systems increasingly improve themselves by editing skills: prompts, rubrics, plans, tool contracts, examples, validators, and traces.

By Sridhar Mahadevan

llms agents benchmarks safety

arXiv AI

Jul 14

When Sensing Varies with Contexts: Context Probing for Tactile Few-Shot Class-Incremental Learning

arXiv:2603. 25115v2 Announce Type: replace Abstract: Few-shot class-incremental learning (FSCIL) aims to recognize novel classes from only a few labeled samples while retaining previously learned knowledge.

By Yifeng Lin, Aiping Huang, Wenxi Liu, Si Wu, Tiesong Zhao, Zechao Li, Zheng-Jun Zha

rag benchmarks

arXiv AI

Jul 14

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts

arXiv:2508. 10123v3 Announce Type: replace-cross Abstract: Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT).

By Maxime Heuillet, Yufei Cui, Boxing Chen, Audrey Durand, Prasanna Parthasarathi

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 14

Optimizing ARDL Models for Retail Sales Forecasting and Fair Pricing

arXiv:2607. 09956v1 Announce Type: new Abstract: Pricing food products to balance profitability with consumer welfare is a central challenge for retailers.

By Sujay Uday Rittikar

benchmarks safety

arXiv Machine Learning

Jul 14

RecRec: Recursive Refinement for Sequential Recommendation

arXiv:2607. 10541v1 Announce Type: cross Abstract: Sequential recommender systems typically infer user preferences through single-pass encoding of interaction histories without iterative refinement, relying on increasingly deep architectures to capture complex patterns.

By Pervez Shaik, Prosenjit Biswas, Abhinav Thorat, Ravi Kolla, Niranjan Pedanekar

benchmarks

arXiv Machine Learning

Jul 14

LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation

arXiv:2607. 10546v1 Announce Type: new Abstract: Discovering governing partial differential equations (PDEs) from noisy observational data is a fundamental challenge in scientific machine learning.

By Jinyang Du, Hao Ma, Xiaohu Shi, Bo Yang, Yanchun Liang, Heow Pueh Lee, Chunguo Wu

llms benchmarks

arXiv Machine Learning

Jul 14

SKooP: Symmetric Koopman Predictions for Faster and More Generalizable Legged Robot Locomotion with Reinforcement Learning

arXiv:2607. 11624v1 Announce Type: cross Abstract: Reinforcement learning (RL) algorithms classically suffer from poor sample efficiency.

By Evelyn D'Elia, Weishu Zhan, Giulio Turrisi, Giulio Romualdi, Giuseppe L'Erario, Raffaello Camoriano, Wei Pan, Daniele Pucci

agents reinforcement-learning robotics benchmarks

arXiv AI

Jul 14

Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points

arXiv:2607. 11363v1 Announce Type: cross Abstract: Text-based evaluations of Theory of Mind (ToM) in Large Language Models (LLMs) often involve cognitive tests akin to the Sally-Anne task that can be gamed due to exposure to relevantly similar tasks in pre-training and do not obviously test models' functional ToM abilities in ways that generalize to naturalistic settings.

By Roberta Rocca, Sami Boukortt, Geoff Keeling, Winnie Street

llms benchmarks

arXiv Machine Learning

Jul 14

Reliability Scaling Laws for Quantized Large Language Models

arXiv:2607. 10855v1 Announce Type: new Abstract: Quantization is a powerful strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters.

By Sirine Ayadi, S\'andor Dar\'oczi, Stephan G\"unnemann, Bertrand Charpentier

llms efficiency benchmarks

arXiv Machine Learning

Jul 14

Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

arXiv:2607. 09999v1 Announce Type: cross Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.

By Renuka Oladri, Mohan Vamsi Varadaraju Priya, Jerry Wu

llms efficiency benchmarks

arXiv Machine Learning

Jul 14

Physics-Aware Conditional SetGAN for Spatially Consistent Multi-User TR 38.901 Channel Generation

arXiv:2607. 11429v1 Announce Type: new Abstract: TR 38.

By Mauro Gonzalo Tarazona-Levano, David Lopez-Perez, Nicola Piovesan, David Gomez-Barquero

diffusion benchmarks

arXiv AI

Jul 14

Where Experts Disagree, Models Fail: Detecting Implicit Legal Citations in French Court Decisions

arXiv:2603. 22973v2 Announce Type: replace Abstract: Applying computational methods to law at scale requires separating genuine legal reasoning from surface similarity.

By Avrile Floro (UPHF), Tamara Dhorasoo (UPHF), Soline Pellez (UPHF), Nils Holzenberger

benchmarks

arXiv AI

Jul 14

Looped State-Space Language Models with Adaptive Exit-State Selection

arXiv:2607. 10110v1 Announce Type: new Abstract: Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters.

By Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa

llms robotics benchmarks

arXiv AI

Jul 14

UNIT: Unleash Large Language Models Potential for Graph Continual Learning

arXiv:2607. 10159v1 Announce Type: new Abstract: In real-world multimodal web scenarios, graph-structured data often arrives in a streaming manner, making graph continual learning a crucial paradigm for continuously modeling such evolving structures.

By Tairan Huang, Yili Wang, Beibei Hu, Yiting Shi, Qiutong Li, Changlong He, Jianliang Gao

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jul 14

MeloBottleneck: Self-Supervised Melody Skeleton Extraction with a Latent Subsequence Bottleneck

arXiv:2607. 10233v1 Announce Type: cross Abstract: Melody skeleton extraction aims to derive a shorter melody that preserves structural notes while removing ornaments.

By Fan Bu, Rongfeng Li, Linfeng Fan

benchmarks safety

arXiv Machine Learning

Jul 14

When Classical Baselines Are Tuned as Carefully as the Quantum Model, Does Quantum Reservoir Computing Still Win?

arXiv:2607. 09905v1 Announce Type: cross Abstract: Can a small quantum computer forecast a changing signal better than an ordinary classical method?

By Tushar Pandey

benchmarks

arXiv Machine Learning

Jul 14

SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering

arXiv:2607. 10413v1 Announce Type: cross Abstract: Prototype-based Incomplete Multi-view Clustering has recently attracted increasing attention by exploiting prototypes as semantic anchors for missing-view imputation.

By Yaoyuan Guo, Zhibin Gu, Songhe Feng, Yuhui Zheng, Bing Li

benchmarks

arXiv AI

Jul 14

TopoExplore: Topological Discrimination for Archive-Based Exploration

arXiv:2607. 09971v1 Announce Type: new Abstract: Archive-based exploration methods such as Go-Explore select which visited state to return to using visitation rarity, and frontier methods return to the boundary of the unknown; neither asks whether the unexplored region behind a boundary is enterable at all.

By Jason Carlson

benchmarks

arXiv AI

Jul 14

PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

arXiv:2604. 22823v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) rely on multimodal pre-training over diverse data sources, where different datasets often induce complementary cross-modal alignment capabilities.

By Zibo Shao, Baochen Xiong, Xiaoshan Yang, Yaguang Song, Qimeng Zhang, Haifeng Chen, Changsheng Xu

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jul 14

Forecasting Generative Amplification

arXiv:2509. 08048v4 Announce Type: replace-cross Abstract: Generative networks are perfect tools to enhance the speed and precision of LHC simulations.

By Henning Bahl, Sascha Diefenbacher, Nina Elmer, Tilman Plehn, Jonas Spinner

benchmarks

Get the daily briefing