arXiv AI September 7, 2026 By Xiaodong Li, Peiwei Liu

When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models

The paper investigates numerical hallucination in financial language models, comparing a base instruction-tuned model, a domain-adapted model (FT‑A), and a numeracy-enhanced domain model (FT‑A+B+C). A three-level detectability taxonomy is introduced, and the study finds that domain fine-tuning dramatically increases hallucination rates—up to 98%—and that numeracy supervision actually amplifies hallucination. Template injection is identified as a key mechanism, leading the authors to conclude that numerical hallucination stems from degraded numerical restraint rather than lack of reasoning, and they recommend comprehensive evaluation and grounding-aware deployment practices.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Aug 31

Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMs

arXiv:2606.21359v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses signific...

By Raia Abu Ahmad, Nikolas Rauscher, Ekaterina Borisova, Fabio Barth, Georg Rehm, Sebastian M\"oller

llms fine-tuning benchmarks safety

arXiv AI

Sep 30

The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining

arXiv:2609.32964v2 Announce Type: replace Abstract: Language models (LMs) often hallucinate by committing to confident answers rather than abstaining, even when they do not have enough information to...

By Vy Nguyen, Ziqi Xu, Jeffrey Chan, Estrid He, Feng Xia, Renqiang Luo, Erik Cambria, Xiuzhen Zhang

llms benchmarks safety

arXiv AI

Jun 11

From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data

arXiv:2606. 07537v1 Announce Type: cross Abstract: Large language models hallucinate--producing fluent, confident, factually wrong outputs--with a consistency that persists across generations and scales.

By Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem

llms safety

arXiv AI

Jul 31

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

arXiv:2607. 26368v1 Announce Type: cross Abstract: Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in qualitatively different ways.

By Aman Kumar, Lasitha Vidyaratne, Dipanjan D Ghosh, Arnab Chakrabarti, Ahmed K Farahat

llms rag fine-tuning benchmarks

arXiv AI

Sep 7

Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning

The study investigates whether large language models (LLMs) can generalize arithmetic reasoning across different input formats, such as numeric and verbal expressions in English, Spanish, and Italian. By using attribution patching, the researchers identified the specific internal circuits each model employs for numeric versus verbal arithmetic tasks. They found that the degree of overlap between a model’s numeric circuit and its verbal circuit predicts how well the model generalizes to verbal formats, explaining variations in performance across languages and formats without requiring labeled data.

By Andrea Gregor de Varda, Sana Pandey, Pengrui Han, Jacob Andreas, Evelina Fedorenko

llms

Hugging Face Trending Papers

Jul 29

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in qualitatively different ways. Detecting a conflict is only the first step: review workflows may also need to determine its type, since numerical, temporal, referential, factual, and normative inconsistencies require different evidence and downstream checks.

llms rag fine-tuning benchmarks

One email a morning, machine-written

Related stories

Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMs

The Commit-Abstain Circuit: Why Language Models Hallucinate Instead of Abstaining

From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text