The paper presents an empirical study of factual errors in human-written text, focusing on corrections in newspaper articles to build a taxonomy of common mistakes such as kanji misconversions and unit errors. It evaluates large language models’ ability to detect these errors, finding that even advanced models like GPT‑5.4 achieve only a 52% word‑level F1 score on synthetic data, underscoring the difficulty of the task. The work highlights the gap in research on factual error detection in human writing compared to LLM hallucinations.
Large language models (LLMs) frequently produce confident yet factually incorrect responses when user inputs contain misleading premises, a phenomenon we attribute to fact perturbations in the input....
The paper introduces DEDUCE, a three‑stage framework that turns large language models into proactive error correctors by detecting input fact errors, devising correction strategies, and delivering reliable answers. It also presents MisFactQA, a dataset of factual errors, and new metrics for robustness evaluation. Experiments on TruthfulQA, FalseQA, and MisFactQA show significant gains in accuracy and error correction across Qwen, LLaMA, and Gemma models.
SWORD is a new benchmark that tests large language models’ ability to reject factually incorrect statements across eight major languages by distorting Wikidata triples. The benchmark reveals that models often perform better on semantically plausible distortions than on random ones, indicating a reliance on distributional familiarity rather than true factual verification. It also shows significant performance drops for East Asian languages, with gaps up to 28 percentage points, highlighting asymmetric multilingual factual reasoning capabilities.
arXiv:2608.29307v1 Announce Type: cross Abstract: Language models increasingly answer questions by consulting retrieved documents rather than memory alone, a design now common in search assistants an...
arXiv:2609.23573v1 Announce Type: new Abstract: Synthetic essays can help reduce dependence on human-written data in Automated Essay Scoring (AES). However, they often lack realistic errors, limiting...
arXiv:2602.11166v2 Announce Type: replace-cross Abstract: Parameter-efficient fine-tuning (PEFT) methods are widely used to adapt large language models (LLMs) to downstream tasks and are often assume...
arXiv:2609.20838v1 Announce Type: new Abstract: In this study, we examine how modern LLMs generate and detect fake news under controlled settings across four manipulation scenarios. These are open-en...
FLAWS is a new benchmark that tests large language models’ ability to find and pinpoint errors that invalidate key claims in scientific papers. The dataset contains 713 paper‑error pairs created by inserting claim‑invalidating mistakes into peer‑reviewed articles using LLMs, and it includes an automated metric for measuring error detection and localization. Five state‑of‑the‑art LLMs were evaluated, with GPT‑5 achieving the highest accuracy of 39.1% when selecting the top 10 candidate error texts.
The paper investigates how adjectival modifiers affect the semantic plausibility of events, using the Adept benchmark of 16,000 English sentence pairs that differ by a single adjective. Experiments show that sentence transformers, despite being conceptually suited to the task, underperform compared to models like RoBERTa. The authors provide an error analysis and discuss the implications of their findings for future work on balancing training and test data.
The paper investigates why supervised AI‑text detectors, specifically a RoBERTa‑based model, can be fooled by subtle changes in language. By applying semantic, structural, and tokenizer‑level perturbations to a large dataset and controlled Mistral‑7B‑Instruct outputs, the authors show that increasing verb diversity makes machine text easier to detect and that detection scores correlate with statistical complexity, leading to a high false‑positive rate on formal human writing. They also evaluate an event‑based latent space detector, finding that paraphrasing and homoglyphs significantly alter extracted event sequences and verbs, yet the detector’s performance remains modest (AUC 0.577).
arXiv:2609.15561v1 Announce Type: new Abstract: Evaluating the factual correctness of large language models (LLMs) is vital for many applications. But are our evaluation tools themselves trustworthy?...
Large language models (LLMs) can generate fluent Arabic answers, yet factual errors remain difficult to detect, localize, explain, and verify. Existing hallucination benchmarks often provide response-level labels, with limited support for identifying the exact erroneous content, explaining why it is incorrect, or selecting the correct factual answer.