arXiv Computation and Language August 27, 2026 By Zihao Zheng, Baichuan Li, Junyi Yao, Jiayu Long

Reliable Financial Named Entity Recognition Under Domain Shift: Confidence Estimation and Selective Prediction

Read the original on arXiv Computation and Language →

The paper investigates confidence estimation and selective prediction for financial named entity recognition (NER) under domain shift, using a stress test across SEC filings, financial news, and social media. It evaluates BERT and LoRA‑tuned Qwen2.5 models with five inference‑time confidence signals, finding that whole‑output probability is a strong in‑domain error detector but weak out‑of‑domain, while entity‑span probability and self‑consistency remain robust. Abstention can dramatically reduce sentence error on high‑confidence in‑domain data, but offers limited benefit under extreme social‑media shift, suggesting a staged deployment that first detects severe distribution shift before applying confidence gating.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

Aug 21

Reliable Financial Named Entity Recognition under Domain Shift

arXiv:2608. 19558v1 Announce Type: new Abstract: Financial AI systems often train information extractors on one textual register and deploy them across filings, news, and user-generated content, while standard F1 scores do not indicate which predictions remain safe to automate when the input distribution changes.

By Zihao Zheng, Baichuan Li, Junyi Yao, Jiayu Long

nlp fine-tuning

arXiv AI

Jun 17

The Stanford EDGAR Filings Dataset: Reconstructing U.S. Corporate and Financial Disclosures into Layout-Faithful and Token-Efficient Pretraining Data

arXiv:2606. 18192v1 Announce Type: new Abstract: As high-quality public web corpora become increasingly exhausted, clean long-context documents have become a scarce and expensive source of training data for large language models (LLMs).

By Nick Bettencourt, Xiaowei Ding, Kay Giesecke

llms computer-vision benchmarks

arXiv Machine Learning

Aug 6

From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models

arXiv:2608. 04200v1 Announce Type: cross Abstract: Financial sentiment classifiers are commonly evaluated against human labels, but strong linguistic performance does not necessarily imply economically useful return predictability.

By Fusheng Luo

llms fine-tuning benchmarks

arXiv AI

Jun 12

Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings

arXiv:2602. 07294v4 Announce Type: replace-cross Abstract: With the increasing deployment of Large Language Models (LLMs) in the finance domain, LLMs are increasingly expected to parse complex regulatory disclosures.

By Yidong Jiang, Junrong Chen, Eftychia Makri, Jialin Chen, Peiwen Li, Ali Maatouk, Leandros Tassiulas, Eliot Brenner, Bing Xiang, Rex Ying

llms rag benchmarks safety

arXiv Machine Learning

Aug 20

Converting Expert Deliberation into Financial Signals Through A Context-Aware NLP Pipeline

The paper presents the CDSP (context-conditional deliberation signal pipeline), which transforms investment committee meeting transcripts into structured predictive features. CDSP segments transcripts into topical chunks, assigns asset‑class context labels via a large language model, maps financial keywords to a taxonomy, and adds sentiment polarity and mention frequency features. Using these engineered features on 48 monthly meetings, the best model—combining sentence embeddings with CDSP features—achieves 73% accuracy and a 0.73 F1 score, outperforming a simple stock‑choice baseline, though the improvement is not statistically significant.

By Vivek Batra, Kristin Chen, Sanjiv Das, Samuel Judge, Harshad Khadilkar, Sukrit Mittal, Amir Nasrollahzadeh, Daniel Ostrov, Jacob Sisk

llms rag nlp

Hugging Face Trending Papers

Jul 29

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in qualitatively different ways. Detecting a conflict is only the first step: review workflows may also need to determine its type, since numerical, temporal, referential, factual, and normative inconsistencies require different evidence and downstream checks.

llms rag fine-tuning benchmarks