arXiv AI

Mind the Gap: Can Frontier LLMs Pass a Standardized Office Proficiency Exam?

arXiv:2606. 10956v1 Announce Type: new Abstract: The deployment of Large Language Model (LLM) agents for computer automation is accelerating, yet their ability to navigate complex, professional-grade productivity software is largely untested.

arXiv AI

Jun 10

Flaws in the LLM Automation Narrative

arXiv:2606. 11166v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly described as performing at the level of human experts on knowledge economy tasks.

By George Perrett, Javae Elliott, Jennifer Hill, Marc Scott

llms benchmarks

arXiv AI

Jul 15

SheetMind: An End-to-End LLM-Powered Multi-Agent Framework for Spreadsheet Automation

arXiv:2506. 12339v2 Announce Type: replace-cross Abstract: We present SheetMind, a modular multi-agent framework powered by large language models (LLMs) for spreadsheet automation via natural language instructions.

By Xi Cheng, Ruiyan Zhu, Ke Liu, Rakesh Chowdary Machineni, Lyuhao Chen, Brian Zhu, Daniel Jin, Zheng Qi, Neeraj Parihar, Zhoutian Xu, Oliver Gao

llms agents benchmarks safety

Hugging Face Trending Papers

Aug 6

Benchmarking and Enhancing LLMs for Rule-Intensive Review of National Standard Documents

Large language models (LLMs) increasingly support complex professional tasks, yet their capabilities in rule-intensive document review remain insufficiently evaluated. National standard documents, such as China GB/T standards, offer a representative testbed: they are lengthy, highly structured, and governed by explicit rules for scope, terminology, normative wording, and cross-section consistency.

llms agents nlp benchmarks safety

arXiv AI

Jul 3

Office Comprehension Benchmark

arXiv:2607. 01245v1 Announce Type: cross Abstract: We introduce Office Comprehension Bench (OCB), the first public benchmark to jointly evaluate LLM systems on Word, Excel, and PowerPoint comprehension over native file formats (.

By Firoz Shaik, Mateus Pican\c{c}o Lima Gomes, Tanvir Aumi, Jingci Wang, Milos Milunovic, Filip Basara, Ivana Jovanovic, Vishwas Suryanarayanan, Neha Nandan Kenkare, Weiyao Xie, Zhipeng Han, Zheng Zhang, Waleed Shahid, Jay Rathi, Russell Scherer, Thong Q. Nguyen, Michael Bentley, Tamara Stankovic, Rasika Chakravarthy, Vishal Chowdhary

llms benchmarks

arXiv AI

1d ago

PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows

arXiv:2607. 06008v3 Announce Type: replace Abstract: While Large Language Model (LLM) agents excel at monolingual long-horizon planning and tool use, enterprise workflows inherently require processing multilingual resources across extended trajectories.

By Hongliang Li, Yijin Liu, Zhiwei Zhang, Zihe Liu, Xinyue Lou, Jinan Xu, Fandong Meng, Kaiyu Huang

llms agents benchmarks

arXiv AI

Jun 18

CAPRA: Scaling Feedback on Software Architecture Deliverables with a Multi-Agent LLM System

arXiv:2606. 18976v1 Announce Type: cross Abstract: Automated assessment in software engineering education has advanced significantly for code grading and essay scoring.

By Marco Becattini, Niccol\`o Caselli, Matteo Minin, Roberto Verdecchia, Enrico Vicario

llms agents multimodal safety

arXiv Machine Learning

Jun 8

Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios

arXiv:2606. 06546v1 Announce Type: new Abstract: Evaluating large language models (LLMs) for education requires measuring how models teach, not only what they know.

By Tao Liu, Ye Lu, Ruohua Zhang, Siyu Song, Wentao Liu, Aimin Zhou, Hao Hao

llms agents benchmarks safety

Hugging Face Trending Papers

Jun 29

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making. However, most existing spreadsheet benchmarks evaluate isolated operations such as single-formula generation or local cell edits, and therefore fail to capture end-to-end workflows in realistic business settings.

llms agents benchmarks

arXiv AI

Jul 31

IDP AutoOpt: Agent-Driven Optimization of Document Processing Pipeline Configurations

arXiv:2607. 26075v1 Announce Type: cross Abstract: We present IDP AutoOpt, an autonomous LLM agent that discovers high-performing configurations for intelligent document processing (IDP) pipelines.

By David Kaleko, Sergey Ivanov, Md Mofijul Islam

llms rag agents computer-vision benchmarks

arXiv AI

Jun 30

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

arXiv:2606. 29955v1 Announce Type: cross Abstract: Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making.

By Jian Zhu, Yuzheng Zhang, Zeyao Ma, Bohan Zhang, Armin Schoepf, Daniel Woloch, Peter Yiliu Wang, Guangyu Robert Yang, Samuel Jacob, Siddharth Nagisetty, Abhiram Chundru, Jean Lin, Spencer Mateega, Jing Zhang

llms agents benchmarks

arXiv AI

Aug 12

DOCSCHISEL: Adaptive Tool Documentation Optimization Framework for LLM Agents

arXiv:2608. 10037v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly rely on external tools to accomplish complex real-world tasks, making tool documentation a critical grounding resource for LLM agents.

By You Lu, Kun Zhang, Bihuan Chen, Xin Peng

llms agents benchmarks

arXiv AI

Jun 24

Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System

arXiv:2606. 24839v1 Announce Type: new Abstract: Agentic data analysis systems produce rich outputs, including code, numerical results, and verbal diagnostics.

By Tian Zheng, Kai-Tai Hsu

llms agents

Related stories