Spreadsheets and tables are widely used representations for structured data analysis, but effective analysis still requires substantial manual effort and domain expertise. Recent large language model (LLM) agents can automate parts of this process, but they often provide limited transparency into intermediate decisions, rely on implicit assumptions, struggle with multi-table comparison, and repeat similar workflows without adapting to a user's preferences.
The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.
By Alexander Gill, Md Farhan Ishmam, Xuyen Nguyen, Neha Bhat, Parker Henry DeYoung, Fateme Hashemi Chaleshtori, Nathan Stringham, Kenneth Marino, Ana Marasovi\'c
AdaLens is an interactive system designed to monitor and steer long-running, autonomous data analysis workflows powered by large language models. It provides a storyline-based interface that unifies analytical plans, execution progress, intermediate findings, and data-column involvement, enabling analysts to observe evolving reasoning and evidence. The system also offers steering interactions that allow users to redirect low-value directions or deepen promising ones during execution.
By Yangtian Liu, Yan Miao, Shuhan Liu, Yunfan Zhou, Dae Hyun Kim, Di Weng, Yingcai Wu
arXiv:2608. 16045v1 Announce Type: cross Abstract: LLM-based data-analysis tools are increasingly used to help users analyze messy spreadsheets and workbooks, from answering questions over uploaded files to generating code, summaries, and visualizations.
By Yike Yuan, Virum Ranka, Tina Lasisi, Lin Ma
arXiv:2607. 01647v1 Announce Type: cross Abstract: Data science aims to derive actionable insights from heterogeneous raw data, unlocking the value of the massive amounts of data generated in modern society.
By Zhaoyan Sun, Shan Zhong, Daizhou Wen, Jiaxing Han, Guoliang Li, Ying Yan, Peng Zhang, Yu Su, Xiang Qi, Baolin Sun, Chengyuan Yang, Tao Fang, Huaiyu Ruan
RefactorPlatform is an open‑source harness that standardizes the evaluation of repository‑scale refactoring agents by fixing the environment and systematically varying design choices such as model backbone, execution regime, and prompt specificity. Each run operates in an isolated workspace, logs detailed telemetry, and verifies changes with AST‑based checks. Experiments on 100 RefactorBench tasks show that AST‑aware chunking improves performance by 25‑30%, a lean retrieval‑augmented single agent outperforms a sub‑agent configuration, and retrieval’s accuracy gains offset its token overhead, keeping cost per successful refactoring unchanged.
By Aziz Ben Amor, Drish Mali, Mann Acharya, Vijayasri Iyer, S\'ebastien Brati\`eres