arXiv AI
Prospect Theory in Large Language Models: A Meta-Analysis of Loss Aversion, Diminishing Sensitivity, and Probability Weighting
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 10
KTO: Model Alignment as Prospect Theoretic Optimization
The paper introduces KTO, a new loss function that aligns large language models with human preferences by directly maximizing a prospect-theoretic utility function rather than log-likelihood of preferences. It builds on the idea that existing alignment objectives implicitly use human-aware losses (HALOs) that reflect biases such as loss aversion. KTO achieves performance comparable to or better than preference-based methods across 1B to 30B parameter models while only requiring a binary desirability signal, and it highlights that no single HALO is universally best, emphasizing the importance of choosing the right inductive biases for a given setting.
arXiv AI
Jun 19
Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact
arXiv:2606. 20205v1 Announce Type: new Abstract: Psychological instruments designed for humans are increasingly used to assign large language models (LLMs) stable psychological profiles that affect their usability, safety assessment, and use as proxies for human participants in research.
arXiv AI
Oct 19
Compression Precedes, Content Decides: Emergent Misalignment in Reward Models
The paper investigates emergent misalignment in reward models, showing that fine‑tuning reward models to prefer harmful content in a single domain (medical or financial) causes the preference to generalize to seven other domains. The authors fine‑tune two state‑of‑the‑art reward models—Skywork‑Reward‑V2‑Llama‑3.1‑8B and Skywork‑Reward‑V2‑Qwen3‑8B—and find that the models still perform well on the standard RewardBench 2 evaluation while exhibiting a preference for harm. Mechanistically, the study identifies two key changes in the reward models’ representations: a geometric compression that reduces the angle between sound and harmful responses, and a content‑specific rotation that aligns harmful representations more closely with the scalar head, thereby increasing their reward score.
arXiv AI
Jul 7
Out-of-Distribution Generalization of Risk Aversion in Language Models
arXiv:2607. 02755v1 Announce Type: cross Abstract: Training AIs to be risk-averse in resources could offer a failsafe in the event that AIs turn out misaligned.
arXiv Computation and Language
Oct 8
The Long Road to the Same Answer: Cognitive Bias Under Escalating Reasoning Budgets in Large Language Models
The study investigates whether large language models (LLMs) that allocate extra computation during inference—termed reasoning models—reduce classic decision biases compared to their non‑reasoning counterparts. Using 30 vignettes covering six cognitive biases and varying token budgets up to 8,192 tokens, the authors find that reasoning models are not less biased, and increased deliberation does not reliably diminish bias magnitude. Only anchoring showed a bias in the human direction, while other biases either remained unchanged or moved further from human patterns, suggesting that test‑time reasoning does not guarantee rationality.
arXiv AI
Jul 28
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
arXiv:2508. 08992v4 Announce Type: replace Abstract: Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human behavior under such uncertainty.