Was this person ever at that place, and if so, when? Answering such questions from noisy, multilingual historical documents is the central challenge of HIPE-2026, the third edition of the HIPE evaluation series.
arXiv:2602. 17663v3 Announce Type: replace Abstract: HIPE-2026 is a CLEF evaluation lab dedicated to person-place relation extraction from noisy, multilingual historical texts.
arXiv:2606. 31325v1 Announce Type: new Abstract: We present HistoriQA-ThirdRepublic: a French-language dataset of multi-hop historical questions derived from parliamentary debates and newspapers of the French Third Republic.
The paper introduces CLAW-4L, a benchmark of 300 pairs of English and non‑English Wikipedia biographies (French, Chinese, Azerbaijani) focused on women from non‑English contexts, complete with claim annotations and a fine‑grained claim‑pair relation corpus. It proposes a claim‑based enrichment framework that extracts claims from both biographies, aligns them to identify enrichment evidence from the non‑English version, and rewrites the English biography accordingly. Experiments demonstrate that non‑English Wikipedia biographies can improve English biography coverage, though lower‑resource settings still pose challenges.
The paper introduces Multi-Negative Direct Preference Optimisation (MDPO), a pairwise objective that compares the correct entity with all valid rejected candidates for each mention, extending the single-negative approach used in prior work. MDPO retains the Bradley‑Terry formulation of Direct Preference Optimisation while leveraging the full candidate set through masked, length‑normalised sequence scores. Experiments on French, German, English, Swedish, and Finnish historical newspaper datasets (hipe‑2020 and newseye) show that MDPO outperforms both supervised fine‑tuning and single‑negative DPO, especially for NIL mentions, semantic ambiguity, OCR noise, and historically challenging names, and highlight candidate retrieval as a key bottleneck.
Event analysis is an essential and fundamental direction of information extraction, involving various event-centric tasks at different granularity of documents. While large language models (LLMs) have preliminarily achieved promising performance in part of these tasks individually, their capability in event analysis still lacks comprehensive understanding due to restricted document granularity, task designs, and data source of existing benchmarks.
TRACE is a training‑free, agentic retrieval framework that enables accountable source discovery in historical archives, addressing challenges such as OCR degradation and genre heterogeneity. Developed for the DECIDON project on French Third Republic political discourse, it is deployed internally for 24 researchers across six institutions. On the HistoriQA‑ThirdRepublic benchmark, TRACE achieves R@10 of 0.856 and MRR of 0.653, outperforming sparse, dense, graph‑based, and other agentic RAG baselines, especially on multi‑hop and cross‑corpus questions, while costing only about $0.02 per question.
The paper proposes a new way to evaluate language models on cultural understanding by focusing on interpretive depth rather than just factual recall. It argues that literary interpretation, where scholars can disagree yet still assess the quality of evidence, provides a useful framework for testing how models handle cultural references, reuse, and transformation across texts. The authors suggest linking evidence-centered benchmarks, preserving scholarly disagreement, and conducting model-development experiments on literary data, with Danish literature as a starting point for broader applications.
arXiv:2608.23507v1 Announce Type: cross Abstract: Historical people may appear under different languages, scripts, and transcription traditions, while distinct individuals may share highly similar or...
arXiv:2607. 08143v1 Announce Type: cross Abstract: We present the results of HIPE-OCRepair-2026, an ICDAR competition on LLM-assisted OCR post-correction of historical documents.
CMNIE is a new benchmark for extracting structured information from Chinese military news, covering event triggers, arguments, named entities, and entity relations under a unified schema. The dataset contains 13,000 manually annotated instances with 7 event types, 10 argument roles, 7 entity types, and 8 relation types. Experiments show that current supervised models, zero‑shot LLMs, and fine‑tuned LLMs struggle with relation extraction and exact span matching, highlighting the challenge of joint structured extraction in this domain.
The paper introduces a weakly supervised framework for extracting dataset mentions from forced displacement and Fragile, Conflict, and Violence (FCV) documents. It uses a lightweight model trained on general research literature to generate candidate mentions, which are then refined by a large language model that validates or rejects them and corrects boundaries. The refined annotations are augmented with synthetic and contrastive examples to fine‑tune the model, achieving 74.1% precision and 70.5% recall on a benchmark of 1,706 passages, with higher precision (89.5%) on passages that contain dataset references.
arXiv:2606. 27881v1 Announce Type: cross Abstract: Temporal variation poses a unique challenge for named entity recognition (NER) in historical texts, where entities drift in surface form and salience across time.