ShredBench is a new benchmark for evaluating multimodal large language models (MLLMs) on the task of reconstructing documents from shredded fragments. It uses an automated pipeline that generates fragmented documents from Markdown, covering English, Chinese, code, and table scenarios with varying fragmentation levels. Experiments show that while MLLMs perform well on intact documents, their performance drops sharply when documents are shredded, revealing a gap in fine‑grained cross‑modal reasoning.
By Zichun Guo, Yuling Shi, Wenhao Zeng, Chao Hu, Haotian Lin, Terry Yue Zhuo, Jiawei Chen, Xiaodong Gu, Wenping Ma
Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction, and image captioning. This separation leaves scie...
arXiv:2610.20335v1 Announce Type: new
Abstract: Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction,...
By Yuheng Chen, Yufan Chen, Zhuojun Cai, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Weijia Fan, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen
arXiv:2608. 15064v1 Announce Type: new Abstract: Parsing visual documents into machine-readable representations is fundamental to document intelligence.
By Yuefeng Zou, Yichen Lu, Jingxiao Yang, Bingtao Fu, Gaoyang Zhang, Xiongfei Bai, Tian Chen, Xiang Qi
The paper introduces an empirical attribution framework to pinpoint failures in multi‑page visually rich document understanding systems. By applying controlled interventions in a unified document question‑answering pipeline, the authors analyze how evidence representation, selection, and usage affect performance. Experiments on MMLongBench‑Doc reveal that complementary modalities can mitigate conversion errors, that adding non‑gold pages can both correct and break answers, and that abstention strategies trade unsupported answers for false refusals, with varying effects across different backbones.
By Lewei Xu, Yihao Ding, Zihan Xu, Daniel Yitian Su, Daochang Liu, Siwen Luo, Yifan Peng, Wei Liu
arXiv:2606. 09578v1 Announce Type: new Abstract: Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored.
By Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee, Roy Ka-Wei Lee, Preslav Nakov