The paper evaluates eight open‑source lightweight Vision‑Language Models (up to 7 B parameters) for converting document images into structured JSON, focusing on heritage collections with historical handwriting and non‑standard layouts. It compares zero‑shot, few‑shot, and fine‑tuned settings, measuring performance with CER, ANLS*, and mAP‑F1, and examines the effects of hyperparameter tuning, image preprocessing, and multi‑stage training. The study demonstrates that adapted VLMs can serve as a sustainable, private alternative to commercial OCR systems for institutional archives.
By Uddipan Basu Bir, Vincent Christlein, Andreas Maier, Mathias Zinnen
The FAIR Digital Object (FDO) framework mandates that metadata attribute values be expressed as persistent identifiers (PIDs) wherever possible, to produce a fully machine-actionable graph in which ev...
arXiv:2608.23263v1 Announce Type: new
Abstract: The FAIR Digital Object (FDO) framework mandates that metadata attribute values be expressed as persistent identifiers (PIDs) wherever possible, to pro...
By Zeyd Boukhers, Lingxiao Kong, Xenophon Zabulis, Georgios Toubekis
The paper introduces an AI-assisted pipeline for retrieving goldsmith marks from silverware images, combining mark localization with metric‑learning fine‑tuning on three backbone models (ResNet‑50, ViT‑S/16, and DINOv2 ViT‑S/14). Systematic tests of cropping strategies show that manual cropping and metric‑learning fine‑tuning yield the best performance, with DINOv2 ViT‑S/14 achieving 62.63% mAP and 73.74% Top‑1 accuracy. The authors release a manually annotated dataset, code, and a public web interface to support reproducibility and adoption in digital humanities.
By Atmik Tiwari, Vincent Christlein, Mark Fichtner, Freya Gohlke, Birgit Sch\"ubel, Theresa Witting, Heike Zech, Mathias Zinnen
The paper introduces AttnPrint, a white‑box fingerprinting method that extracts low‑frequency components of cross‑modal attention distributions to identify multimodal large language models (MLLMs). It also presents DistillTrace, a black‑box auditing tool that uses hypothesis testing of MLLM outputs to detect potential model infringement. Experiments on 154 model instances across 19 architectures show that AttnPrint effectively detects derivative models and remains robust to downstream modifications, while DistillTrace reveals distillation relationships under various techniques.
By Chao Huang, Meng Tong, Kejiang Chen
Public institutions hold large volumes of sensitive documents and support tickets that cannot leave the premises, ruling out cloud-hosted language models entirely. We report on RAGAL, a retrieval-augmented assistant for the technical-support team of AFIR, the Romanian Agency for Financing Rural Investments, built and operated under three hard constraints: zero data egress (no external API calls, even for synthetic data), a read-only mandate (the assistant drafts, humans execute), and a single 8 GB consumer laptop as the only development and training machine.
arXiv:2606. 10217v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly fine-tuned on domain-specific datasets that may contain sensitive, dataset-level properties.
By Pengrun Huang, Chhavi Yadav, Ruihan Wu, Kamalika Chaudhuri
arXiv:2607. 11889v1 Announce Type: cross Abstract: Large language models trained on unrestricted internet corpora inevitably embed information from the future, introducing lookahead bias that compromises the validity of backtests and causal inference in finance and the social sciences.
By Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu
MUDIDI is a two-stage framework designed to digitize multilingual dictionaries that are currently only available as scanned images. The first stage assesses character recognition and markup preservation, while the second stage segments dictionary entries and maps them into the SIL Multi-Dictionary Formatter schema. The authors also release a dataset of 30 annotated dictionaries and benchmark OCR, LLM, and VLM systems, finding that LLMs generally outperform others and that providing additional context improves digitization quality.
By David Setiawan, Temuulen Khishigsuren, Milind Agarwal, Pagnarith Pit, Aso Mahmudi, Ekaterina Vylomova
arXiv:2606. 09648v1 Announce Type: cross Abstract: Multi-modal data management has emerged as a central research topic in the database community, spanning data integration, semantic query processing, and data quality assessment.
By Luciano Duarte, Olga Ovcharenko, Sebastian Schelter
arXiv:2608.20868v1 Announce Type: cross
Abstract: Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information ext...
By A. Said Gurbuz (IBM Research Zurich, ETH Zurich), Ahmed Nassar (IBM Research Zurich), Christoph Auer (IBM Research Zurich), Maksym Lysak (IBM Research Zurich), Lucas Morin (IBM Research Zurich), Matteo Omenetti (IBM Research Zurich), Tim Strohmeyer (IBM Research Zurich), Panagiotis Vagenas (IBM Research Zurich), Nikolaos Livathinos (IBM Research Zurich), Michele Dolfi (IBM Research Zurich), Peter Staar (IBM Research Zurich)
GoDeep is an annotation‑free method for open‑vocabulary 3D scene understanding that uses a vision‑language model solely as a translator to generate structured, entity‑level descriptions of each image. These descriptions are projected and aggregated in a language‑only embedding space, eliminating the need for a 3D training corpus or domain‑specific encoder. The approach achieves competitive performance on ScanNet++ and a cultural heritage benchmark, accurately localizes out‑of‑vocabulary objects, and offers explainable, point‑level predictions.
By Thodoris Betsas, Anastasios Doulamis, Andreas Georgopoulos