arXiv AI

TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs

arXiv:2606. 09578v1 Announce Type: new Abstract: Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored.

arXiv AI
Aug 28

A Table Is Worth 64 Tokens: Pixel-level Compression for Multi-Table Document Question Answering

The paper investigates pixel-level table compression for document question answering, comparing five vision‑language models across two benchmarks and varying visual‑token budgets. It finds that representing tables as native‑resolution images matches text in performance and efficiency, while highly downscaled images still allow the model to identify relevant tables but lose readability, leading to longer reasoning traces. A two‑step, training‑free method first selects relevant tables from compressed images and then reasons over them at native resolution, saving 41% of tokens and improving accuracy by 7 points over single‑step native‑resolution QA, while using 15% fewer tokens than the most efficient single‑step compressed setup without accuracy loss.

By I\~nigo Alonso, Mirella Lapata
arXiv Machine Learning
Jun 9

IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation

arXiv:2601. 04498v2 Announce Type: replace Abstract: Infographics are composite visual artifacts that combine data visualizations with textual and illustrative elements to communicate information.

By Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, Luoxuan Weng, Xiuqi Huang, Minfeng Zhu, Yingchaojie Feng, Yuyu Luo, Wei Chen
arXiv Machine Learning
Sep 4

Towards Universal Tabular Embeddings: A Benchmark Across Data Tasks

The paper introduces TEmBed, a unified benchmark for evaluating tabular embeddings across four representation levels—cell, row, column, and table—using a diverse set of models. It demonstrates that the best model depends on the specific task and representation level, providing practical guidance for selecting embeddings in real-world applications. The study aims to facilitate the development of more general-purpose tabular representation models.

By Liane Vogel, Kavitha Srinivas, Niharika D'Souza, Sola Shirai, Oktie Hassanzadeh, Horst Samulowitz
arXiv AI
Jun 8

MCERF: Advancing Multimodal LLM Evaluation of Engineering Documentation with Enhanced Retrieval

arXiv:2604. 09552v2 Announce Type: replace-cross Abstract: Engineering rulebooks and technical standards contain multimodal information like dense text, tables, and illustrations that are challenging for retrieval augmented generation (RAG) systems.

By Kiarash Naghavi Khanghah, Hoang Anh Nguyen, Anna C. Doris, Amir Mohammad Vahedi, Daniele Grandi, Faez Ahmed, Hongyi Xu
arXiv Machine Learning
Jul 9

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.

By Miguel Lopez-Duran, Elena Marrero, Julian Fierrez, Marta Robledo-Moreno, Ruben Vera-Rodriguez, Daniel DeAlcala, Aythami Morales, Ruben Tolosana, Oscar Delgado, Alvaro Ortigosa, Javier Ortega-Garcia
arXiv Computation and Language
Sep 1

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.

By Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
arXiv AI
Jun 16

VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA

arXiv:2606. 16092v1 Announce Type: cross Abstract: Real-world documents combine text with tables, charts, photographs, and diagrams arranged in diverse layouts, yet existing research on multimodal large language models (MLLMs) for document QA predominantly produces text-only responses, underutilizing these visual elements.

By Young Rok Jang, Hyesoo Kong, Kyunghwan An, Jae Sub Huh, Gyeonghun Kim, Stanley Jungkyu Choi