Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,561 stories · RSS feed

arXiv Machine Learning

Jun 15

Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models

arXiv:2512. 04981v2 Announce Type: replace-cross Abstract: Text-to-image (T2I) systems increasingly rely on Large Language Model (LLM)-based text conditioning to interpret and expand user prompts.

By NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim

llms rag diffusion benchmarks safety

arXiv Machine Learning

Jun 15

Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models

arXiv:2606. 14222v1 Announce Type: new Abstract: The rapid evolution of Time Series Foundation Models (TSFMs) has advanced zero-shot forecasting across diverse domains.

By Xilin Dai, Yiding Liu, Hongjie Xia, Yifan Hu, Zewei Dong, Jiang-Ming Yang, Qiang Xu

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 15

Arbitrary control over multimode wave propagation for machine learning

arXiv:2402. 17750v2 Announce Type: replace-cross Abstract: Controlled multimode wave propagation can enable more space-efficient photonic processors than architectures based on discrete components connected by single-mode waveguides.

By Tatsuhiro Onodera, Martin M. Stein, Benjamin A. Ash, Mandar M. Sohoni, Melissa Bosch, Ryotatsu Yanagimoto, Marc Jankowski, Timothy P. McKenna, Tianyu Wang, Gennady Shvets, Maxim R. Shcherbakov, Logan G. Wright, Peter L. McMahon

benchmarks

arXiv AI

Jun 15

CADET: Physics-Grounded Causal Auditing and Training-Free Deconfounding of End-to-End Driving Planners

arXiv:2606. 14438v1 Announce Type: cross Abstract: End-to-end (E2E) autonomous-driving planners trained by imitation are prone to statistical shortcuts: they associate scene elements that merely co-occur with expert actions (a roadside object, a building facade) with driving decisions, rather than the variables that causally determine them.

By Zikun Guo

agents benchmarks

arXiv AI

Jun 15

Same-Origin Policy for Agentic Browsers

arXiv:2606. 14027v1 Announce Type: cross Abstract: Agentic browsers integrate autonomous AI agents into web browsers, enabling users to accomplish web tasks through natural-language instructions.

By Xilong Wang, Xiaoxing Chen, Patrick Li, Dawn Song, Neil Gong

agents benchmarks

arXiv Machine Learning

Jun 15

Decoupled Latent Optimization of Diffusion Models for Full Waveform Inversion

arXiv:2606. 14139v1 Announce Type: new Abstract: Full waveform inversion (FWI) recovers subsurface velocity from seismic recordings by solving a severely ill-posed, nonconvex PDE-constrained optimization.

By Chen Min, Zheng Ma

diffusion benchmarks

arXiv AI

Jun 15

VISTA: View-Consistent Self-Verified Training for GUI Grounding

arXiv:2606. 14579v1 Announce Type: new Abstract: When applying Group Relative Policy Optimization (GRPO) for GUI Grounding, rollouts are sampled from a single screenshot view; groups often become either all failures on difficult instances or all successes on easy ones, yielding no useful relative advantage.

By Xinyu Qiu, Yunzhu Zhang, Heng Jia, Shuheng Shen, Changhua Meng, Linchao Zhu

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 15

When to Write and When to Suppress: Route-Specialized Dual Adapters for Memory-Assisted Knowledge Editing

arXiv:2606. 14668v1 Announce Type: new Abstract: Knowledge editing systems must update selected facts while preserving nearby but irrelevant behavior.

By Yining Huang

llms rag fine-tuning benchmarks

arXiv AI

Jun 15

Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders

arXiv:2503. 19947v2 Announce Type: replace-cross Abstract: Generalized metric depth understanding is critical for precise vision-guided robotics, which current state-of-the-art (SOTA) vision-encoders do not support.

By Paul Koch, J\"org Kr\"uger

diffusion computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 15

From Sorting Algorithms to Scalable Kernels: Bayesian Optimization in High-Dimensional Permutation Spaces

arXiv:2507. 13263v4 Announce Type: replace-cross Abstract: Bayesian Optimization (BO) is a powerful tool for black-box optimization, but its application to high-dimensional permutation spaces is severely limited by the challenge of defining scalable representations.

By Zikai Xie, Linjiang Chen

benchmarks

arXiv Machine Learning

Jun 15

AGORA: Can Deliberation and Governance Gates Absorb Participation Bias in Transit Planning?

arXiv:2606. 13696v1 Announce Type: cross Abstract: Transit network design depends not only on the optimization algorithm but also on who shows up to the public hearing.

By Jung-Hoon Cho, Cathy Wu

agents benchmarks safety

arXiv Machine Learning

Jun 15

Temporally Consistent Graph Q-Networks for Intelligent Network Control

arXiv:2606. 13848v1 Announce Type: cross Abstract: Mobile networks continue to grow in complexity and next generation networks are expected to support both increasing traffic loads and more diverse services.

By Zacharias Veiksaar, Maxime Bouton

agents reinforcement-learning benchmarks

arXiv AI

Jun 15

3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding

arXiv:2603. 04976v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards ( RLVR ) has emerged as a transformative paradigm for enhancing the reasoning capabilities of Large Language Models ( LLMs), yet its potential in 3D scene understanding remains under-explored.

By Xiongkun Linghu, Jiangyong Huang, Baoxiong Jia, Siyuan Huang

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 15

How do Self-Supervised Remote Sensing Vision Models Transfer to Downstream Tasks?

arXiv:2606. 13896v1 Announce Type: cross Abstract: Self-supervised geospatial foundation models (GeoFMs) learn transferable representations from remote sensing data, but their downstream behavior is difficult to characterize.

By Julia Romero, Qin Lv, Morteza Karimzadeh

llms rag computer-vision fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 15

Trend-Aware Multi-Task Learning for Short-Term Energy Forecasting

arXiv:2511. 09789v3 Announce Type: replace Abstract: Short-term energy forecasting plays an important role in real-time operational decision-making, such as electricity market bidding and power system dispatch, where both numerical accuracy and correct directional signals are essential.

By Fulong Yao, Wanqing Zhao, Chao Zheng, Xiaofei Han

benchmarks

arXiv Machine Learning

Jun 15

Robin-Neumann Coupling of PINN and FEM Solvers: A Steklov-Poincar\'e View, with Application to Fluid-Structure Interaction with Contact

arXiv:2606. 14181v1 Announce Type: cross Abstract: Physics-informed neural networks (PINNs) are meshless and carry moving geometry and topology change through resampling of collocation points; the finite-element method (FEM) is the workhorse for boundary-fitted discretisations.

By Mikel Landajuela

benchmarks

arXiv AI

Jun 15

Capability Minimization as a Safety Primitive: Risk-Aware Causal Gating for Least-Privilege LLM Agents

arXiv:2606. 13884v1 Announce Type: new Abstract: Modern decision systems increasingly rely on learned components whose outputs may be confident yet wrong, exposing downstream actions to costly errors.

By Laxmipriya Ganesh Iyer, Rahul Suresh Babu

llms agents benchmarks safety

arXiv AI

Jun 15

Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL

arXiv:2606. 14211v1 Announce Type: new Abstract: LLMs are increasingly deployed as agents that interact with external environments and observe feedback such as execution results, error messages, and tool outputs.

By Yinglun Zhu

llms agents reinforcement-learning benchmarks data-engineering

arXiv AI

Jun 15

Causal Object-Centric Models for Planning with Monte Carlo Tree Search

arXiv:2606. 14418v1 Announce Type: new Abstract: We introduce COMET (Causal Object-centric Model for Efficient Tree search), a model-based reinforcement learning algorithm that performs Monte Carlo Tree Search in a slot-structured latent space.

By Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik, Aleksandr Panov

llms diffusion reinforcement-learning benchmarks safety

arXiv AI

Jun 15

From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI

arXiv:2606. 14502v1 Announce Type: new Abstract: Large Language Models (LLMs) are undergoing a fundamental transformation from conversational generators into integrated AI systems capable of reasoning, action, memory, and self-improvement.

By Yongheng Zhang, Ziang Liu, Jiaxuan Zhu, Shuai Wang, Xiangqi Chen, Haojing Huang, Jiayi Kuang, Siyu Chen, Ao Shen, Hao Wu, Qiufeng Wang, Qian-Wen Zhang, Junnan Dong, Wenhao Jiang, Ying Shen, Hai-Tao Zheng, Yinghui Li, Di Yin, Xing Sun, Philip S. Yu

llms agents reinforcement-learning benchmarks

Get the daily briefing