arXiv Machine Learning

Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness

arXiv:2608. 10008v1 Announce Type: cross Abstract: LLM recommenders for top-$K$ item suggestion regularly emit titles outside the target catalog.

arXiv AI

Aug 13

Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching

arXiv:2605. 29055v2 Announce Type: replace Abstract: This paper describes an approach to hallucination detection and mitigation using a HOPE-inspired Nested Learning architecture with Continuum Memory Systems (CMS) and semantic similarity caching, tested on a hybrid benchmark of 310 prompts (217 epistemic-uncertainty prompts, 93 fabrication-induction stress tests).

By Diego Gosmar, Deborah A. Dahl

llms agents benchmarks safety

arXiv Machine Learning

Aug 13

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

arXiv:2608. 11922v1 Announce Type: cross Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer $F_1$ from 0.

By Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu, Li-Yang Chang, Chuan-Ju Wang

llms rag nlp reinforcement-learning benchmarks

arXiv AI

Jul 8

An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models

arXiv:2606. 09843v3 Announce Type: replace-cross Abstract: Large language models (LLMs) give stable answers to personality questionnaires, yet these self-reports fail to predict how the models behave.

By Juan Manuel Contreras

llms safety

arXiv Machine Learning

Jun 24

Do LLM Attribution Metrics Transfer? Auditing Retrieval-Augmented Generation Evaluation Across Datasets and Constructs

arXiv:2606. 23915v1 Announce Type: cross Abstract: Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable.

By Tianyu Ding, Aditya Nannapaneni, Juan Pablo De la Cruz Weinstein

llms rag

arXiv Machine Learning

Jul 9

Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scale

arXiv:2607. 07670v1 Announce Type: cross Abstract: Large language models hallucinate most about entities they have never seen.

By Grzegorz Brzezinka

llms

arXiv AI

2d ago

Calibrated Trust, Not Sharper Prediction: An Empirical Test of Uncertainty Fusion

arXiv:2608. 14617v1 Announce Type: cross Abstract: A recurring proposal in legal AI is to improve case-outcome prediction by fusing uncertainty tools (evidence graphs with belief propagation, sequential Bayesian odds updating, Dempster-Shafer combination, and conformal prediction) into one pipeline.

By Surya Saka

llms

Hugging Face Trending Papers

Jul 8

Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scale

Large language models hallucinate most about entities they have never seen. We ask whether a model's activations betray entity familiarity before a single answer token is generated, and whether that signal predicts the factual reliability of the answers.

llms

arXiv AI

Jun 10

Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agents

arXiv:2606. 10315v1 Announce Type: cross Abstract: LLM-as-judge is the default instrument for evaluating conversational agents, yet its reliability is almost always reported as agreement with human ratings, not recall of real defects.

By Sawyer Zhang, Alexander Wang, Sophie Lei

llms agents safety

arXiv AI

Aug 3

The Asymmetric Effects of Knowledge Distillation on Bias in Small Language Models

arXiv:2607. 28639v1 Announce Type: cross Abstract: We show that knowledge distillation in small instruction-tuned language models has asymmetric effects on bias.

By Plawan Kumar Rath

llms fine-tuning efficiency safety

arXiv AI

Jun 10

An LLM-Native Psychometric Instrument Does Not Predict LLM Behavior: Evidence Across 25 Models

arXiv:2606. 09843v1 Announce Type: cross Abstract: Large language models (LLMs) produce stable self-reports on personality inventories, but these self-reports do not predict observed behavior.

By Juan Manuel Contreras

llms safety

arXiv Machine Learning

Jul 27

SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation

arXiv:2605. 03534v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds answers in retrieved passages, yet relevance does not guarantee sufficiency: a topical passage may still fail to justify the answer.

By Jingxi Qiu, Zeyu Han, Cheng Huang

llms rag benchmarks safety

arXiv AI

Aug 5

Quantifying Hallucinations in Language Language Models on Medical Textbooks

arXiv:2603. 09986v3 Announce Type: replace-cross Abstract: Hallucinations, the tendency for large language models to provide responses with factually incorrect and unsupported claims, is a serious problem within natural language processing for which we do not yet have an effective solution to mitigate against.

By Brandon C. Colelough, Davis Bartels, Dina Demner-Fushman

llms nlp benchmarks safety

Related stories