Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,859 stories · RSS feed

arXiv AI

Jun 15

A Deep Reinforcement Learning (DRL)-Based Transformer Method for Solving the Open Shop Scheduling Problem

arXiv:2606. 13682v1 Announce Type: new Abstract: The open shop scheduling problem (OSSP) arises in many industrial and service settings but remains computationally challenging as the number of jobs and machines increases.

By Faezeh Ardali, Mwembezi A. Nyelele, Gerald M. Knapp

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 15

Non-Parametric Machine Text Detection via Multi-View Gaussian Processes

arXiv:2606. 14060v1 Announce Type: new Abstract: Adversarial conditions such as paraphrasing and targeted style transfer sharply degrade the accuracy of machine text detectors.

By Aleem Khan, Nicholas Andrews

llms benchmarks safety

arXiv AI

Jun 15

Causal Object-Centric Models for Planning with Monte Carlo Tree Search

arXiv:2606. 14418v1 Announce Type: new Abstract: We introduce COMET (Causal Object-centric Model for Efficient Tree search), a model-based reinforcement learning algorithm that performs Monte Carlo Tree Search in a slot-structured latent space.

By Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik, Aleksandr Panov

llms diffusion reinforcement-learning benchmarks safety

arXiv AI

Jun 15

SAFformer:Improving Spiking Transformer via Active Predictive Filtering

arXiv:2605. 08270v2 Announce Type: replace-cross Abstract: Spiking Neural Networks (SNNs) offer notable advantages in biological plausibility and energy efficiency, making them promising candidates for building low-power Transformers.

By Zequan Xie, Weiming Zeng, Yunhua Chen, Sichang Ling, Tongyang Chen, Jinsheng Xiao

llms benchmarks

arXiv AI

Jun 15

Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL

arXiv:2606. 14211v1 Announce Type: new Abstract: LLMs are increasingly deployed as agents that interact with external environments and observe feedback such as execution results, error messages, and tool outputs.

By Yinglun Zhu

llms agents reinforcement-learning benchmarks data-engineering

arXiv AI

Jun 15

EmoMind: Decoding Affective Captions from Human Brain fMRI

arXiv:2605. 16739v2 Announce Type: replace-cross Abstract: Decoding visual experience from brain activity has advanced substantially, but current brain-to-text systems largely recover semantic content while discarding affect.

By Bilal A. Mohammed, Lin Gu, Ruogu Fang

llms benchmarks

arXiv AI

Jun 15

A Benchmark and Framework for Evaluating Next Action Predictions in Spreadsheets

arXiv:2606. 13802v1 Announce Type: cross Abstract: Predictive code completion greatly accelerates how quickly developers work.

By Tejas Agrawal, Vu Le, Sumit Gulwani, Gust Verbruggen

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 15

Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation

arXiv:2606. 13657v2 Announce Type: replace Abstract: On-policy distillation (\textsc{OPD}) has recently become a prominent post-training recipe by combining two desirable ingredients: on-policy student trajectories and dense teacher supervision.

By Guo Yu, Wenlin Liu, Yulan Hu, Hao-Xuan Ma, Jun-Peng Jiang, Han-Jia Ye

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jun 15

Decompose Sparsely Where You Should, Absorb Densely Where You Should No

arXiv:2606. 14040v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition.

By Ruixuan Deng, Zehao Jin, Zekun Wang, Zihan Dong

efficiency benchmarks safety

arXiv AI

Jun 15

Same-Origin Policy for Agentic Browsers

arXiv:2606. 14027v1 Announce Type: cross Abstract: Agentic browsers integrate autonomous AI agents into web browsers, enabling users to accomplish web tasks through natural-language instructions.

By Xilong Wang, Xiaoxing Chen, Patrick Li, Dawn Song, Neil Gong

agents benchmarks

arXiv Machine Learning

Jun 15

Persona-Pruner: Sculpting Lightweight Models for Role-Playing

arXiv:2606. 14695v1 Announce Type: new Abstract: Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user persona.

By Jinsu Kim, Jihoon Tack, Noah Lee, Jongheon Jeong

llms efficiency benchmarks

arXiv AI

Jun 15

Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources

arXiv:2606. 14141v1 Announce Type: cross Abstract: Sound events are entities with semantic identities, locations, and trajectories, but current audio-language models usually reason about clips as global event content.

By Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida, Kim Sung-Bin, Toshimitsu Uesaka, Takashi Shibuya, Kyeongyoon Lee, Tae-Hyun Oh, Yuki Mitsufuji

llms benchmarks

arXiv AI

Jun 15

Succeeding at Scale: Enterprise Retrieval Benchmark Construction and Index-Preserving Query Adaptation for Multi-Tenant Search

arXiv:2601. 04646v4 Announce Type: replace-cross Abstract: Large-scale multi-tenant retrieval systems generate extensive query logs but lack curated relevance labels for effective domain adaptation, resulting in substantial underutilized "dark data.

By Prateek Jain, Shabari S Nair, Ritesh Goru, Prakhar Agarwal, Ajay Yadav, Yoga Sri Varshan Varadharajan, Constantine Caramanis

llms fine-tuning benchmarks

arXiv AI

Jun 15

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost

arXiv:2602. 03120v2 Announce Type: replace-cross Abstract: Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune.

By Yinggan Xu, Kajetan Schweighofer, Risto Miikkulainen, Xin Qiu

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jun 15

Fractured Chain-of-Thought Reasoning

arXiv:2505. 12992v4 Announce Type: replace-cross Abstract: Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.

By Baohao Liao, Hanze Dong, Yuhui Xu, Doyen Sahoo, Christof Monz, Junnan Li, Caiming Xiong

llms benchmarks

arXiv Machine Learning

Jun 15

Aligned but Stereotypical? How System Prompts Shape Demographic Bias in LLM-Based Text-to-Image Models

arXiv:2512. 04981v2 Announce Type: replace-cross Abstract: Text-to-image (T2I) systems increasingly rely on Large Language Model (LLM)-based text conditioning to interpret and expand user prompts.

By NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim

llms rag diffusion benchmarks safety

arXiv AI

Jun 15

UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities

arXiv:2504. 20734v5 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has shown substantial promise in improving factual accuracy by grounding model responses with external knowledge relevant to queries.

By Woongyeong Yeo, Kangsan Kim, Soyeong Jeong, Jinheon Baek, Sung Ju Hwang

rag fine-tuning benchmarks

arXiv AI

Jun 15

An integrated interpretable control effectiveness learning and nonlinear control allocation methodology for overactuated aircrafts

arXiv:2606. 13794v1 Announce Type: cross Abstract: Nonlinear dynamics and the strong couplings that arise between multiple effectors undermine the assumptions behind conventional, linear control allocation techniques.

By Umut Demir, Aamir Ahmad, Walter Fichter

benchmarks safety

arXiv Machine Learning

Jun 15

Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems

arXiv:2606. 14155v1 Announce Type: new Abstract: Context adaptation automates prompt engineering in LLM-based systems by iteratively revising tunable prompts from task feedback, without modifying model weights.

By Tan Zhu, Tong Yao, Kananart Kuwaranancharoen, Amit Singh, Yushang Lai, Deepa Mohan, Shankara Bhargava

llms agents benchmarks

arXiv AI

Jun 15

VISTA: View-Consistent Self-Verified Training for GUI Grounding

arXiv:2606. 14579v1 Announce Type: new Abstract: When applying Group Relative Policy Optimization (GRPO) for GUI Grounding, rollouts are sampled from a single screenshot view; groups often become either all failures on difficult instances or all successes on easy ones, yielding no useful relative advantage.

By Xinyu Qiu, Yunzhu Zhang, Heng Jia, Shuheng Shen, Changhua Meng, Linchao Zhu

reinforcement-learning benchmarks

Get the daily briefing