← Back to all news
arXiv AI By Jiahui tang, Kuicai Dong, Dexun Li, Hongchao Gu, Haocheng Yu, Wei Han, Chen Zhang, Yong Liu, Hao Wang, Enhong Chen

DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

Read the original on arXiv AI →

DEEPCHART is a new benchmark that evaluates large language models (LLMs) on faithful data‑science chart generation. It contains 1,482 expert‑annotated instances from scientific papers, financial filings, and ecosystem reports, and assesses chart creation through an Extract–Reason–Visualize pipeline. Experiments show that while LLMs can produce visually plausible charts, they frequently hallucinate data at the extraction and reasoning stages, especially in long, noisy, and multimodal contexts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 2

Faithful Chart Generation for Multimodal Deep Research: Frame-Evidence Co-Adaptation

The paper introduces Frame‑Evidence Co‑Adaptation (FECA), an evidence‑adaptive visual planning framework for generating analytical charts in multimodal deep research. FECA treats chart generation as an iterative interaction between visual frames and retrieved evidence, allowing frames to be guided, revised, or dropped based on evidence availability. Experiments on 100 real‑world research topics demonstrate that FECA improves numerical fidelity while maintaining report quality and chart utility.

By Yuxin Yue, Yingchen Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Xueqi Cheng
multimodal
More like this →
arXiv AI
Oct 1

ChartDensity-Bench: Benchmarking MLLMs for Numerical Data Reconstruction under Visual Density

ChartDensity-Bench is a benchmark designed to evaluate multimodal large language models (MLLMs) on their ability to reconstruct structured numerical data from scientific charts that vary in visual density. The benchmark uses charts paired with source-level ground-truth data and systematically changes the number of simultaneously presented charts (k = 1, 3, 6, 9) to assess how density affects reconstruction performance. A multi‑dimensional evaluation framework measures structural reliability, reconstruction completeness, parseability, and numerical fidelity, revealing that numerical reconstruction generally worsens as visual density increases, with varying degrees of degradation across different models.

By Xinhe Wu, Yadong Jin
llmsnlpmultimodalbenchmarks
More like this →
arXiv AI
Jun 10

ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering

arXiv:2510. 04514v3 Announce Type: replace Abstract: Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts-those requiring precise visual interpretation rather than relying on textual shortcuts.

By Rachneet Kaur, Nishan Srishankar, Zhen Zeng, Sumitra Ganesh, Manuela Veloso
llmsagentsnlpmultimodalbenchmarks
More like this →