The paper examines how fine‑tuning large language models (LLMs) for specific human values can unintentionally affect other, untrained values and broader behaviors. By adapting four instruction‑tuned LLMs to Hofstede’s value dimensions, the authors evaluate cross‑value transfer and observe side effects on risk preferences, social cognition, and safety traits. They conclude that post‑training value alignment should include comprehensive behavioral assessments and that further research is needed to determine the optimal placement of value training within the development pipeline.
arXiv:2605. 30036v2 Announce Type: replace Abstract: Large Language Models (LLMs) demonstrate a remarkable capacity to adopt different personas and roles; however, it remains unclear whether they can manifest behavior that adheres to a coherent, human-like value structure.
The paper investigates whether fine‑tuning large language models (LLMs) on value survey responses can shift their value‑conditioned outputs and downstream behavior. Researchers first estimate baseline value profiles for several open‑source LLMs using the ten Schwartz values, then fine‑tune the models on survey‑style supervision that selectively alters target value ratings while preserving non‑target values. The resulting shifts are evaluated on held‑out survey questions and on out‑of‑domain behavioral proxies, such as a contextual moral‑judgment benchmark from Reddit AITA posts and a text‑based interactive decision‑making benchmark. The study finds that survey‑based supervision can change both survey responses and downstream behavior, but the transfer is strongly model‑ and value‑dependent and influenced by how value descriptions are framed.
The paper investigates whether large language models (LLMs) possess intrinsic value systems and how to quantify and align them. By projecting responses from 106 LLMs and 95,000 human survey profiles into a shared sociological space, the authors confirm that LLMs do have values, though these values form a concentrated, idealized core rather than mirroring human diversity. They introduce the Prior-Environment-Cognition (PEC) framework to mathematically define value expression and propose an adaptive Alignment Prescription that identifies minimal interventions—ranging from prompts to targeted parameter updates—to steer LLM values efficiently without harming general performance.
arXiv:2510. 26707v2 Announce Type: replace-cross Abstract: As LLMs occupy an increasingly important role in society, they are more and more confronted with questions that require them not only to draw on their general knowledge but also to align with certain human value systems.
arXiv:2609.37914v1 Announce Type: cross Abstract: Fine-tuning large language models on narrow, misaligned tasks can undo their post-training alignment and induce novel misaligned behaviors -- a pheno...
arXiv:2605. 28591v2 Announce Type: replace-cross Abstract: The validity of AI safety evaluations depends on models behaving consistently across controlled and deployment settings.
The paper introduces the task of predicting alignment generalization, which measures how fine‑tuning a language model to follow a specific value affects its behavior across a broad set of other values. Through a large‑scale study of 66 values, the authors show that representations derived from model activations when applying values in context outperform textual description‑based methods, achieving a correlation of 0.45 versus 0.05. They further demonstrate that these representations can predict downstream robustness and help construct a taxonomy of LLM values based on empirical generalization dynamics.
arXiv:2602. 03160v2 Announce Type: replace Abstract: Aligning Large Language Models (LLMs) with the diverse spectrum of human values remains a central challenge: preference-based methods often fail to capture deeper motivational principles.
arXiv:2607. 22676v1 Announce Type: new Abstract: Post-training is a key mechanism for adapting large language models to downstream tasks.
arXiv:2608.23476v1 Announce Type: new Abstract: Narrow fine-tuning on small, domain-specific datasets can produce broad and surprising changes in model behavior-a phenomenon called weird generalizati...
arXiv:2602. 07356v2 Announce Type: replace Abstract: Aligning large language models (LLMs) with human values has become increasingly important as their influence on human behavior and decision-making expands.
arXiv:2609.39701v1 Announce Type: new Abstract: Value steering should change an LLM's normative priorities while preserving the scenario, facts, and task constraints underlying its answer. Convention...