arXiv AI
Prompt PerturbAItions: Mechanisms of LLM Response Inconsistencies
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 15
How Semantically Stable Are LLM Refusals? Measuring Confusion in Local Safety Boundaries
The paper introduces Semantic Confusion to assess how consistently large language models refuse similar prompts. It presents ParaGuard, a 10k‑prompt corpus of controlled paraphrase clusters, and proposes three token‑level metrics—Confusion Index, Confusion Rate, and Confusion Depth—to measure contradictory refusal decisions across meaning‑preserving paraphrases. Experiments show that global false rejection rates can mask local inconsistencies, revealing that refusal evaluation must consider both frequency and consistency across nearby paraphrases.
arXiv AI
Oct 12
Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs
The paper investigates how safety‑aligned large language models (LLMs) can still be tricked by jailbreaks or over‑refuse benign requests. By analyzing residual‑stream updates in decoder‑only Transformers, the authors find that harmfulness is more readily decoded from attention updates—especially in early layers—while the MLP component is primarily responsible for writing refusal responses. Post‑training enhances MLP‑driven refusal writing without improving harmfulness decodability, and causal interventions confirm that downstream MLP updates are crucial for the refusal shift caused by upstream attention changes. Building on these insights, the authors propose Sentinel, a system that combines an attention‑derived perception score with an MLP‑derived decision score to better detect jailbreaks and over‑refusals across multiple models.
arXiv AI
Jun 8
When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations
arXiv:2606. 07237v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in healthcare for tasks such as clinical question answering, diagnosis support, and report summarization.
arXiv AI
Jul 1
Shared Lexical Task Representations Explain Behavioral Variability In LLMs
arXiv:2604. 22027v2 Announce Type: replace-cross Abstract: One of the most common complaints about large language models (LLMs) is their prompt sensitivity -- that is, the fact that their ability to perform a task or provide a correct answer to a question can depend unpredictably on the way the question is posed.
Hugging Face Trending Papers
Jun 22
The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
Recent advancements in Large Language Models (LLMs) have enabled sophisticated reasoning and content generation, yet their inherent stochasticity poses significant challenges for ensuring predictive credibility. While traditional uncertainty taxonomy paradigms, such as the dichotomy of aleatoric and epistemic uncertainties, provide conceptual foundations, they often fail to capture the multi-component and multi-stage nature of LLM generation and struggle to evaluate the effectiveness of various Uncertainty Quantification (UQ) methods.
arXiv Machine Learning
Sep 3
Prompting the Unknown: Understanding Response Uncertainty in Large Language Models
The paper introduces a prompt-response concept model that links the amount of task-relevant information in a prompt to the uncertainty of responses generated by large language models (LLMs). It identifies four sources of response uncertainty—prompt underspecification, model quality, task variability, and semantic redundancy—and demonstrates that uncertainty decreases as prompt informativeness or model quality increases, analogous to epistemic uncertainty in probabilistic models. Experiments on real-world datasets confirm the theoretical predictions and validate the model.