arXiv AI

ESF-Bench: Benchmarking Challenging Slot-Filling Scenarios for Real-World Enterprise Applications

arXiv:2607. 23326v1 Announce Type: new Abstract: The rapid rise of large language models (LLMs) has driven transformative adoption across enterprises.

arXiv Machine Learning

Jun 9

OpenCompass: A Universal Evaluation Platform for Large Language Models

arXiv:2605. 19276v3 Announce Type: replace-cross Abstract: In recent years, the field of artificial intelligence has undergone a paradigm shift from task-specific small-scale models to general-purpose large language models (LLMs).

By Maosong Cao, Kai Chen, Haodong Duan, Yixiao Fang, Zhiwei Fei, Tong Gao, Ge Jiaye, Mo Li, Hongwei Liu, Junnan Liu, Yuan Liu, Chengqi Lyu, Han Lyu, Ningsheng Ma, Zerun Ma, Yu Sun, Zhiyong Wu, Linchen Xiao, Zhuozhi Xiong, Jun Xu, Haochen Ye, Zhaohui Yu, Yike Yuan, Songyang Zhang, Yufeng Zhao, Fengzhe Zhou, Peiheng Zhou, Dongsheng Zhu, Lin Zhu, Jingming Zhuo

llms benchmarks

arXiv AI

Jun 9

Harmonia: End-to-End RAG Serving Optimization

arXiv:2505. 07833v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) improves the reliability of large language models by integrating external knowledge, but serving RAG pipelines efficiently is challenging because requests traverse heterogeneous components spanning LLM inference, databases, and CPU-side processing.

By Saurabh Agarwal, Bodun Hu, Luis Pabon, Myungjin Lee, Jayanth Srinivasa, Aditya Akella

llms rag efficiency

arXiv AI

Jul 28

E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios

arXiv:2607. 23722v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as agents that interact with stateful environments over multiple steps: gathering hidden information, composing tool calls, and committing state changes.

By Weihuang Zheng, Tianyuan Zou, Eileen Ye, Alphet Liu, Youyong Kong, Ya-Qin Zhang, Duran Zheng, Maxm Pan

llms agents benchmarks

arXiv Computation and Language

Aug 25

SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation Evaluation

arXiv:2608.22390v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong potential in graphical user interface (GUI) generation, but reliable evaluation remains challengi...

By Jiarui Dong, Yin Cai, Zhouhong Gu, Chenmou Wu, Ci Tao, Yiran Chen, Jialing Li, Xiaoran Shi, Juntao Zhang, Zhijun Fang

llms benchmarks

arXiv AI

Sep 1

CORE-T: COherent REtrieval of Tables for Text-to-SQL

arXiv:2601.13111v3 Announce Type: replace-cross Abstract: Realistic text-to-SQL workflows often require joining multiple tables. As a result, accurately retrieving the relevant set of tables becomes...

By Hassan Soliman, Vivek Gupta, Dan Roth, Iryna Gurevych

llms data-engineering

arXiv AI

Jul 10

Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

arXiv:2607. 08057v1 Announce Type: cross Abstract: Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly.

By Jiantong Jiang, Peiyu Yang, Rui Zhang, Feng Liu

llms efficiency

arXiv AI

Jul 14

Algorithm Selection with Zero Domain Knowledge via Text Embeddings

arXiv:2604. 19753v2 Announce Type: replace Abstract: We propose a feature-free approach to algorithm selection: instead of hand-crafted instance features, we use pretrained text embeddings.

By Stefan Szeider

rag benchmarks

arXiv AI

Aug 17

RepBench: Compiling Benchmarks into Capability Representations for Large Language Models

arXiv:2607. 28008v2 Announce Type: replace-cross Abstract: Representation engineering reads and steers capability directions in large language models, yet methods are typically evaluated on paper-specific synthetic data.

By Yanshi Li, Xueru Bai, Shuman Liu, Long Zhang

llms benchmarks

arXiv AI

Jun 30

Beyond IID: How General Are Tabular Foundation Models, Really?

arXiv:2606. 30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.

By Lennart Purucker, Andrej Tschalzev, Nick Erickson, Gioia Blayer, David Holzm\"uller, Alan Arazi, Alexander Pfefferle, Mustafa Tajjar, Ga\"el Varoquaux, Frank Hutter

benchmarks

arXiv AI

Jul 3

AgenticDataBench: A Comprehensive Benchmark for Data Agents

arXiv:2607. 01647v1 Announce Type: cross Abstract: Data science aims to derive actionable insights from heterogeneous raw data, unlocking the value of the massive amounts of data generated in modern society.

By Zhaoyan Sun, Shan Zhong, Daizhou Wen, Jiaxing Han, Guoliang Li, Ying Yan, Peng Zhang, Yu Su, Xiang Qi, Baolin Sun, Chengyuan Yang, Tao Fang, Huaiyu Ruan

llms agents benchmarks

arXiv Machine Learning

Jul 31

SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

arXiv:2607. 27273v1 Announce Type: new Abstract: Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules.

By Jinliang Gao, Ning Yang, Hai Wang, Baili Xiao, Pin Lyu

llms rag reinforcement-learning fine-tuning

arXiv AI

Jul 29

REPREC: Representation Driven Parameter-Efficient Recommendation System

arXiv:2607. 24845v1 Announce Type: cross Abstract: Large language models (LLMs) have been applied to sequential recommendation by formulating it as a natural language task.

By Harshini Kavuru, Dwipam Katariya, Giri Iyengar, Pranab Mohanty, Kalanand Mishra, Kalanand Mishra

llms rag fine-tuning efficiency benchmarks safety

Related stories