The paper introduces an entity‑aware evaluation framework for OCR systems applied to historical documents, focusing on how well names, dates, and identifiers are preserved. Using a benchmark of 175 Berrutti Archive documents and 5,209 manually annotated entity mentions, the authors compare sequence‑recognition OCR, OCR‑specialized vision‑language models, and general‑purpose vision‑language models. Their results show that OCR rankings vary with the evaluation criterion: Qwen 3.6 achieves the highest entity coverage, while some systems recover many entities when small character‑level errors are tolerated, indicating that aggregate transcription metrics alone do not capture meaningful information preservation.
arXiv:2607. 24077v1 Announce Type: cross Abstract: Optical Character Recognition (OCR) is a key component in the digitization of historical archives.
arXiv:2607. 08143v1 Announce Type: cross Abstract: We present the results of HIPE-OCRepair-2026, an ICDAR competition on LLM-assisted OCR post-correction of historical documents.
The paper introduces a controlled benchmark for evaluating large language models (LLMs) on key‑value pair extraction from documents with varying levels of OCR noise. It tests 136 configurations across five instruction‑tuned open‑weight LLMs, three datasets, and four text‑quality conditions, using deterministic decoding to generate 17,688 document‑level inferences. The study finds that clean‑text performance does not reliably predict real‑world robustness, model rankings can reverse under noisy conditions, and few‑shot demonstrations do not always improve accuracy, highlighting reliability risks in OCR‑to‑LLM pipelines.
arXiv:2608. 07917v1 Announce Type: new Abstract: Chinese historical documents preserve valuable cultural heritage, but many collections remain accessible only as scanned page images, preventing full-text retrieval, collation, and computational analysis.
Vision‑Language Models (VLMs) are increasingly replacing traditional OCR for document understanding, but this study shows they often rewrite imperfect text into more plausible forms, a flaw that clean‑text OCR benchmarks miss. The authors created FaithC4, a multilingual perturbation benchmark of 1,455 single‑page documents with scramble, random substitution, and visually similar substitution attacks, and evaluated 15 systems across general‑purpose VLMs, OCR‑specialized VLMs, and traditional OCR pipelines. Results reveal that general‑purpose VLMs suffer up to 6.9 WER points under perturbation, OCR‑specialized VLMs 0.1–3.4 points, and traditional OCR less than 0.8 points on English; probing Qwen3‑VL‑4B shows rewriting occurs only when a perturbed word’s final‑layer representation remains close to the original, with short words (4–6 characters) rewritten up to 10% of the time. whyItMatters":"The findings highlight a critical limitation of VLMs in document transcription, underscoring the need for robust evaluation benchmarks that capture rewriting behavior beyond clean‑text accuracy."
arXiv:2607. 03836v1 Announce Type: cross Abstract: Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natural Language Processing (NLP) capabilities: low-resource transliteration, archaic vocabulary, and noisy input signals.
arXiv:2605.27750v2 Announce Type: replace-cross Abstract: Recent work has shown that Vision-Language Models (VLMs) used for optical character recognition (OCR) can generate plausible but visually uns...
The paper evaluates eight open‑source lightweight Vision‑Language Models (up to 7 B parameters) for converting document images into structured JSON, focusing on heritage collections with historical handwriting and non‑standard layouts. It compares zero‑shot, few‑shot, and fine‑tuned settings, measuring performance with CER, ANLS*, and mAP‑F1, and examines the effects of hyperparameter tuning, image preprocessing, and multi‑stage training. The study demonstrates that adapted VLMs can serve as a sustainable, private alternative to commercial OCR systems for institutional archives.
UniLipi is a unified multi‑script OCR model trained on 13 Indic scripts to recognize handwritten manuscripts under challenging conditions such as varied line geometry, length, and interruptions by non‑textual elements. It uses script‑aware synthetic data generation to perform well even with limited real annotated data. The model also predicts script identity and per‑line character counts, aiding manuscript cataloging, and its representations transfer to contemporary Indic handwriting and several non‑Indic scripts.
SP-DocReader introduces a self‑play framework for OCR that focuses on residual errors after supervised fine‑tuning. It uses Reading Discrepancy Masking to align reference and generated tokens via a longest common subsequence and scores unmatched positions with full conditioning prefixes. The method adds a Focused Fidelity Loss for direct negative log‑likelihood supervision at unmatched ground‑truth positions, training only the OCR module while keeping the backbone frozen, and achieves significant character error rate reductions and ANLS improvements on multiple backbones.
The paper evaluates open-source OCR, LLM, and VLM systems on a high‑risk public sector task: extracting structured data from student application documents. Results show that VLMs generally outperform OCR+LLM pipelines, yet only 4 of 35 configurations achieve F1 scores above 0.5, with most combinations scoring below 0.25. Model size and input quality, especially preserving OCR structure, are critical factors influencing performance.
arXiv:2608. 07917v2 Announce Type: replace Abstract: Chinese historical documents preserve valuable cultural heritage, but many collections remain accessible only as scanned page images, preventing full-text retrieval, collation, and computational analysis.