The paper evaluates eleven vision‑language models (VLMs) for extracting structured fields from business documents, focusing on robustness, cost, and governance rather than just accuracy. Using a held‑out set of 750 synthetic checks, the study finds that fine‑tuning open‑source VLMs on 3,000 samples yields an F1 score above 0.98, surpassing all zero‑shot commercial systems, while GPT‑5 tops the commercial group and Claude Sonnet 4.5 fails on date extraction. The authors also present a practitioner‑oriented selection framework that maps task profiles—such as quality, latency, governance, and volume—to recommended approaches via filtering and total‑cost minimization, demonstrated on a mid‑volume document‑extraction scenario.
By Kushal Patel, Pushkal Shrivastava, Mackenzie Lees, Qirui Lu, Bhargobjyoti Saikia, Liying Li, Junlin Jiang
The paper evaluates open-source OCR, LLM, and VLM systems on a high‑risk public sector task: extracting structured data from student application documents. Results show that VLMs generally outperform OCR+LLM pipelines, yet only 4 of 35 configurations achieve F1 scores above 0.5, with most combinations scoring below 0.25. Model size and input quality, especially preserving OCR structure, are critical factors influencing performance.
By Elias Schubert, Felix Bie{\ss}mann
arXiv:2607. 29677v1 Announce Type: new Abstract: Enterprise workflows increasingly rely on agents for \emph{schema-guided extraction}: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata.
By Boyang Zhang, Adrian Lyjak, Eli Stewart, Zhaoqi Li, Simon Suo
arXiv:2608. 06607v1 Announce Type: cross Abstract: Most document-extraction systems use a single model for all documents.
By Sreerekha Rajendran
The paper evaluates eight open‑source lightweight Vision‑Language Models (up to 7 B parameters) for converting document images into structured JSON, focusing on heritage collections with historical handwriting and non‑standard layouts. It compares zero‑shot, few‑shot, and fine‑tuned settings, measuring performance with CER, ANLS*, and mAP‑F1, and examines the effects of hyperparameter tuning, image preprocessing, and multi‑stage training. The study demonstrates that adapted VLMs can serve as a sustainable, private alternative to commercial OCR systems for institutional archives.
By Uddipan Basu Bir, Vincent Christlein, Andreas Maier, Mathias Zinnen
The study investigates how the choice between processing page images or parsed text in an information extraction pipeline depends on the document’s layout, focusing on privacy‑sensitive, on‑premise scenarios with small models (≤8 B parameters). It evaluates accuracy and energy consumption across input representations, model families, and inference settings, finding that batching dramatically reduces energy use, FP8 quantization offers modest savings, and neural OCR is far more energy‑intensive than classical OCR. The optimal representation varies: vision‑language models excel on layout‑rich documents, while small text‑only models with a cheap parser perform best on near‑plain‑text contracts, achieving higher accuracy and lower energy than any vision‑language setup.
By Christoph Walser, Mauricio Fadel Argerich, Jonathan F\"urst
arXiv:2607. 16203v1 Announce Type: cross Abstract: Document parsing is a foundational step for document understanding tasks such as visual question answering and key information extraction, as it transforms unstructured scanned images into structured representations by extracting textual, visual, and layout information.
By Zihan Xu, Puzhen Wu, Lawrence Chun Man Lau, Wei Liu, Sirui Li, Yifan Peng, Yihao Ding
arXiv:2608.20868v1 Announce Type: cross
Abstract: Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information ext...
By A. Said Gurbuz (IBM Research Zurich, ETH Zurich), Ahmed Nassar (IBM Research Zurich), Christoph Auer (IBM Research Zurich), Maksym Lysak (IBM Research Zurich), Lucas Morin (IBM Research Zurich), Matteo Omenetti (IBM Research Zurich), Tim Strohmeyer (IBM Research Zurich), Panagiotis Vagenas (IBM Research Zurich), Nikolaos Livathinos (IBM Research Zurich), Michele Dolfi (IBM Research Zurich), Peter Staar (IBM Research Zurich)
arXiv:2608. 12133v1 Announce Type: new Abstract: Enterprise guideline documents are heterogeneous and multimodal, combining narrative text, complex tables, and embedded images.
By Shivali Dalmia, Sumukha Thoppanahalli, Mohammadreza Sediqin, Abhishek Mukherji
SAGE is a governed multi‑stage LLM pipeline that transforms enterprise guideline documents—containing narrative text, tables, and images—into structured artifacts. It uses a shared versioned rule store, schema‑validated contracts, and provenance tracking to validate, score, and reconcile extracted rules, automatically approving high‑confidence outputs while flagging uncertain items for human review. In a test on 120 documents, SAGE reduced processing time from days to 20–100 minutes and achieved a 96% success rate with only 3.2% hallucination.
By Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli, Srinivasa Karthikeya Reddy Kovvuri, Abhishek Mukherji
arXiv:2608. 16763v1 Announce Type: new Abstract: Financial document validation in production, such as payroll auditing, tax compliance, and loan underwriting, demands exceptional accuracy, consistency, and reproducibility under strict enterprise constraints.
By Ruoqi Shu, Xuhui Wang, Isaac Wang, Yanming Mai, Bo Wan
The paper presents a method for extracting key information from OCR‑digitized clinical reports, addressing challenges posed by heterogeneous documents and noisy OCR output. It introduces an open key space that is iteratively mined, normalized, clustered, and verified to build a canonical key inventory, and defines key coverage as a metric for inventory completeness. Experiments on reports from over 20 hospitals using a 0.2B BERT model show that performance improves steadily with key coverage, achieving high F1 scores when the top 90 keys are covered and outperforming a fine‑tuned Qwen3‑0.6B baseline.
By Yu Wang, Yingyun Li, Ying Qin, Haiyang Qian