arXiv Machine Learning
Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 28
Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy
arXiv:2607. 22554v1 Announce Type: new Abstract: Large language models (LLMs) often achieve strong accuracy on benchmarks, yet it remains unclear how reliably they apply this knowledge when the same question is phrased in different but equivalent ways.
arXiv AI
Sep 7
A Removal Based Approach to Improve LLM Faithfulness at Test-Time
The paper proposes a test‑time method to enhance the faithfulness of large language model (LLM) explanations by removing concepts not credited in the model’s explanation before re‑querying the model. This approach targets incompleteness—omissions of influential factors—rather than unsoundness, and is model‑agnostic, requiring no changes to model weights. Experiments across two datasets and multiple model families show improved faithfulness compared to standard prompting and faithfulness‑encouraging prompts.
arXiv Machine Learning
Aug 31
Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoning
arXiv:2608.28018v1 Announce Type: cross Abstract: Knowledge-intensive reasoning requires Large Language Models (LLMs) to ground answers in provided evidence. When evidence is insufficient, it is desi...
arXiv AI
Sep 30
Regime Boundary Alignment for Evidence-Gated Question Answering
arXiv:2609.37491v1 Announce Type: cross Abstract: Retrieval-augmented language models are expected to answer from the retrieved evidence, but in practice they often keep answering when that evidence...
arXiv AI
Sep 25
PROOF: Profiling Reliability of Object-Level Facts in Large Language Models
PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.
arXiv AI
Jul 31
Ask don't tell: Reducing sycophancy in large language models
arXiv:2602. 23971v4 Announce Type: replace-cross Abstract: Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts.