arXiv AI
Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
2d ago
Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation
The paper introduces Evidence-Aligned Entity Verification (EAEV), a method for detecting entity-level hallucinations in retrieval-augmented generation (RAG). EAEV aligns generated entities with retrieved evidence across three dimensions and uses counterfactual stability analysis to maintain robust alignments when evidence changes. Experiments on multiple RAG benchmarks show that EAEV consistently outperforms existing hallucination detection methods and generalizes well.
arXiv Computation and Language
1d ago
Probing for Knowledge Attribution in Large Language Models
arXiv:2602.22787v3 Announce Type: replace Abstract: Large language model (LLM) hallucinations, meaning fluent but factually incorrect generations, fall into two types: faithfulness violations, where...
arXiv AI
Jun 9
Evaluating Hallucinations in Domain-Adapted Large Language Models
arXiv:2606. 07521v1 Announce Type: cross Abstract: This study investigates the phenomenon of hallucinations in domain-adapted Large Language Models (LLMs), focusing on the fine-tuning of the Llama-2 model with the Lamini dataset.
arXiv Computation and Language
Aug 25
CALIBURN: Self-Calibrated LLM Unlearning Alignment
CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.
arXiv AI
Aug 24
Knowing but Not Saying: Preventing Factual Access Failures in LLM SFT via Recall-Anchored Distillation
The paper identifies a specific issue in supervised fine‑tuning (SFT) of large language models called factual access failure, where models can recognize correct facts under constrained tests but fail to generate them in open‑ended settings. It demonstrates that SFT can cause both genuine wrong answers and expression‑level errors such as verbosity or formatting mismatches. To mitigate this, the authors propose Recall‑Anchored Distillation (RAD), a self‑distillation method that aligns the fine‑tuned model with the base model’s soft output distribution on unlabeled out‑of‑distribution text, thereby recovering lost factual recall without needing labeled data.
arXiv AI
Sep 1
Small Updates, Big Doubts: Does Parameter-Efficient Fine-tuning Enhance Hallucination Detection ?
arXiv:2602.11166v2 Announce Type: replace-cross Abstract: Parameter-efficient fine-tuning (PEFT) methods are widely used to adapt large language models (LLMs) to downstream tasks and are often assume...