RSIV-Harness is a framework that keeps language model weights fixed while recursively improving the surrounding harness based on failure traces, using a four-layer evaluation to ensure stability. It addresses challenges in open-domain conversational text-to-visualization, such as ambiguous user requests, data source discovery, and iterative chart refinement. The introduced ODAC-Vis benchmark evaluates these capabilities across 640 tasks, and experiments show significant gains in executability and chart structure after a single RSIV-Harness iteration.
By Jiawei Lu, Chen Zhang, Jinwei Lu, Yuanwei Song, Yuanfeng Song, Raymond Chi-Wing Wong
arXiv:2608. 04071v1 Announce Type: new Abstract: Automatically generating professional multimodal reports comprising both textual analysis and visual charts from structured tabular data is a critical challenge in data intelligence.
By Teng Lin, Zhiyang Zhang, Yuyu Luo, Nan Tang
ATP‑Bench proposes a new benchmark for evaluating agentic tool planning in multimodal large language models (MLLMs) that generate interleaved text-and-image responses. The benchmark contains 7,702 QA pairs, including 1,592 visual‑question‑answer pairs, across eight categories and 25 visual‑critical intents, all verified by humans. A Multi‑Agent MLLM‑as‑a‑Judge (MAM) system is introduced to assess tool‑call precision, missed opportunities, and overall response quality without relying on ground‑truth references.
By Yinuo Liu, Zi Qian, Heng Zhou, Jiahao Zhang, Yajie Zhang, Zhihang Li, Mengyu Zhou, Erchao Zhao, Xiaoxi Jiang, Guanjun Jiang
arXiv:2609.26208v1 Announce Type: new
Abstract: Data visualization is central to analytical reasoning, but real-world analysis increasingly requires language-driven interactive interfaces rather than...
By Mizanur Rahman, Aaryaman Kartha, Enamul Hoque Prince
PaperBanana-Interact is a multi-agent system designed to refine scientific diagrams through multi-turn human feedback. The authors introduce MTPaperBananaBench, a benchmark with 292 images and 3,518 user requirements, and a user simulator that generates natural language feedback at each turn. Experiments show that PaperBanana-Interact consistently improves diagram quality, outperforming baseline systems by 11.9–18.6 points and reducing forgetting by 3.7–6.2 points.
By Xueqing Wu, Ashwin Balasubramanian, Bingxuan Li, Dawei Zhu, Kai-Wei Chang, Yale Song, Yiwen Song, Rui Meng, Tomas Pfister, Nanyun Peng
VisCodeBench is a new text‑to‑visualisation benchmark comprising 2,672 samples across educational, scientific, and general‑audience domains, designed to evaluate how well large language models can generate visualisations from natural‑language requests. The study benchmarks three visualisation systems—Codex, Claude Code, and LIDA—paired with various LLMs on seven metrics covering execution, chart content, and visual quality. Results show that while basic execution is near‑solved, no system reliably reproduces reference data, with the best data similarity at 0.81 and dropping to 0.69 on the hardest subset; system design and model choice both significantly influence performance, and latency remains a practical challenge.
By Hy Nguyen, Thomas James Elton, Ella Wilson, Andrew Reeson, Cecile Paris, Josiah Poon, Jonathan K. Kummerfeld
arXiv:2604. 09508v2 Announce Type: replace-cross Abstract: Visual Retrieval-Augmented Generation (VRAG) empowers Vision-Language Models to retrieve and reason over visually rich documents.
By Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, Min Cao
The paper introduces GenUI‑Harness, a multi‑agent system that pairs a Tool Agent for data retrieval with a GUI Coder Agent that generates structured interfaces to aid human‑agent interaction. It tackles training challenges by using Dynamic UX for lightweight reward collection and Reward Auditor to monitor reward quality. The authors present UI‑TAU Bench, a benchmark of 1,300 database‑backed tasks, and show that GenUI‑Harness improves Pass@3 scores and reduces dialogue rounds compared to existing models.
By Xiaolong Li, Xiaohan Xu, Jinyang Li, Xinnuo Xu, Ge Qu, Nan Huo, Jack Williams, Reynold Cheng
IDRBench is a benchmark designed to evaluate the interactive capabilities of deep research agents that use large language models. It introduces controlled opportunities for clarification within a common workflow, comparing autonomous and interactive trajectories by measuring task‑specific report alignment and interaction cost. Experiments on 100 tasks with seven LLMs show that interaction consistently improves alignment, though its effectiveness varies depending on the agents’ questions and feedback integration.
By Yingchaojie Feng, Qiang Huang, Xiaoya Xie, Zhaorui Yang, Jun Yu, Wei Chen, Anthony K. H. Tung
The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.
By Alexander Gill, Md Farhan Ishmam, Xuyen Nguyen, Neha Bhat, Parker Henry DeYoung, Fateme Hashemi Chaleshtori, Nathan Stringham, Kenneth Marino, Ana Marasovi\'c
arXiv:2605. 26144v2 Announce Type: replace-cross Abstract: We present VISTA (VIsual Spec-To-App Benchmark), a benchmark for evaluating the end-to-end web-app generation capabilities of LLM-based agents.
By JunJia Guo (Joe), Yuhang Yao (Joe), Jiawei (Joe), Zhou, Jingdi Chen
arXiv:2512. 11995v2 Announce Type: replace-cross Abstract: While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in most benchmarks, they often struggle in practice with complex open-ended tasks, which usually require multiple rounds of exploration and reasoning in the visual space.
By Chenrui Fan, Yijun Liang, Shweta Bhardwaj, Kwesi Cobbina, Ming Li, Tianyi Zhou