arXiv AI
Jul 23
Information Discernment in Large Language Models
arXiv:2607. 19355v1 Announce Type: new Abstract: LLMs are increasingly used with external knowledge sources like the internet.
Do LLMs Make More Mistakes If They Do Not Believe the Input Data?
The study investigates whether large language models (LLMs) are more prone to errors when they doubt the plausibility of input data, a phenomenon termed context‑memory conflict. Using non‑English and low‑resource language datasets, the authors generate text from factual, counterfactual, and fictional RDF triples in English, Czech, Slovak, and Upper Sorbian, and evaluate faithfulness with both human annotations and an LLM judge (Kimi K3). Contrary to expectations, the results show only a weak context‑memory conflict: counterfactual inputs receive slightly lower faithfulness scores than factual ones, and the choice of LLM judge can significantly affect perceived conflict strength.
Related stories
arXiv AI
Aug 18
When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation
arXiv:2608. 16515v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves large language models by grounding generation in external evidence, but it also introduces a source trust problem: retrieved context may be useful, irrelevant, or even misleading.
arXiv Computation and Language
1d ago
Probing for Knowledge Attribution in Large Language Models
arXiv:2602.22787v3 Announce Type: replace Abstract: Large language model (LLM) hallucinations, meaning fluent but factually incorrect generations, fall into two types: faithfulness violations, where...
arXiv Computation and Language
Sep 1
How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework
The paper introduces a register-aware framework to evaluate how human-like large language models (LLMs) are, focusing on linguistic feature distributions rather than factual correctness. It uses Maximum Mean Discrepancy (MMD) and 67 Biber lexico‑grammatical features to compare LLM‑generated texts with human reference corpora across different registers. Experiments on seven instruction‑tuned, open‑source models across five English datasets show that all LLMs deviate from human baselines, with closeness to human language varying by register and not by model size.
arXiv Computation and Language
2d ago
SWORD: Wikidata-based Distortions Reveal Hidden Cross-Lingual Inconsistencies in LLM Factual Error Rejection
SWORD is a new benchmark that tests large language models’ ability to reject factually incorrect statements across eight major languages by distorting Wikidata triples. The benchmark reveals that models often perform better on semantically plausible distortions than on random ones, indicating a reliance on distributional familiarity rather than true factual verification. It also shows significant performance drops for East Asian languages, with gaps up to 28 percentage points, highlighting asymmetric multilingual factual reasoning capabilities.
arXiv AI
Jun 9
Evaluating Hallucinations in Domain-Adapted Large Language Models
arXiv:2606. 07521v1 Announce Type: cross Abstract: This study investigates the phenomenon of hallucinations in domain-adapted Large Language Models (LLMs), focusing on the fine-tuning of the Llama-2 model with the Lamini dataset.
arXiv Computation and Language
Aug 27
When RAG Fails to Equalize: Geo-bias in Factual Question Answering over Public Companies
The paper investigates whether retrieval‑augmented generation (RAG) can uniformly correct factual errors in large language models (LLMs) by testing six LLMs on a benchmark of about 2,000 public companies. In a controlled factual QA setting, the authors evaluate the models under four conditions—no‑context, perfect context, misleading context, and distraction context—across four atomic attributes. Results show significant geographic disparities in baseline accuracy, and while perfect context improves performance, it does not eliminate these gaps; misleading context often leads models to copy incorrect information, and larger models only marginally reduce structural biases.
arXiv AI
Jun 24
Quantifying Prior Dominance in RAG Systems
arXiv:2606. 23695v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) grounds Large Language Models in external knowledge, yet current evaluations rely on discrete heuristics that suffer from ''epistemic blindness'' - failing to distinguish genuine contextual information extraction from parametric memory recall.
arXiv AI
Jul 10
ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
arXiv:2502. 15543v4 Announce Type: replace-cross Abstract: Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence.
arXiv AI
2d ago
ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic Groups
arXiv:2510.13852v3 Announce Type: replace-cross Abstract: Is an LLM telling you different facts than it's telling me? This paper introduces ConsistencyAI, an independent benchmark for measuring the f...
arXiv AI
Aug 21
Beyond Recall: Behavioral Specification as an Interpretive Layer for AI Personalization
arXiv:2605. 28969v2 Announce Type: replace-cross Abstract: If an AI agent makes decisions on a person's behalf, those decisions must align with its user.
arXiv AI
Jul 23
Internal Knowledge Without External Expression: Probing the Generalization Boundary of a Classical Chinese Language Model
arXiv:2604. 14180v2 Announce Type: replace-cross Abstract: We train a 318M-parameter Transformer language model from scratch on a curated corpus of 1.