← Back to all news
arXiv Machine Learning

Can Revealed Preferences Clarify LLM Alignment and Steering?

The paper proposes an empirical pipeline to estimate the preferences that a large language model (LLM) implicitly optimizes by combining the model’s probability distribution over unknowns with its chosen action, and fitting a discrete choice model to recover the underlying cost function. This revealed-preference framework enables rigorous assessment of whether LLMs act consistently toward a goal, can articulate objectives that align with their decision policy, and can be steered by prompting to follow a user-specified cost function. Experiments across four medical diagnosis domains and various frontier and open-source models show that while many LLMs exhibit internal coherence, they still struggle to accurately report or adopt preferences when guided by users.

Related stories

arXiv AI
Sep 10

When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMs

The paper introduces a decision‑theoretic framework that elicits both probability judgments and decisions from large language models (LLMs) to test whether their reported beliefs are consistent with their actions. It shows that this framework yields empirically testable conditions without assuming a specific utility function. In clinical diagnosis simulations, the authors find that while LLMs’ reported beliefs are not perfect reflections of the information in their decisions, the discrepancies are small for the strongest models.

By Khurram Yamin, Jingjing Tang, Santiago Cortes-Gomez, Amit Sharma, Eric Horvitz, Bryan Wilder
llmsagents
More like this →
arXiv AI
6d ago

Jointly Modeling Preferences and Imperfect Decision-Making in Large Language Models

The paper introduces the Preference-Behavior Model (PBM), which jointly infers Large Language Model (LLM) preferences and imperfect decision-making mechanisms from observed choices. By applying PBM to three social dilemma games—Public Goods, Collective-Risk, and Common-Pool Resource—the authors show that preferences derived from one dilemma can predict behavior in others, whereas traditional behavioral economic elicitation instruments fail to do so consistently. The study also evaluates how well inferred preferences align with persona instructions, revealing both broad agreement and specific divergences across dimensions, LLMs, and games.

By Axel Abels, Elias Fern\'andez Domingos, Tom Lenaerts
llmsagentssafety
More like this →
arXiv AI
5d ago

MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics

MedConstBench is a new medical‑ethics benchmark that treats a model’s deployment constitution as a controllable variable. It is organized around Beauchamp’s four principles and uses human‑calibrated peer review to score responses. The benchmark includes the MedDilemma dataset of clinical dilemmas and demonstrates that different constitutions influence LLM behavior in a dimension‑selective, model‑dependent way, leading to a proposed adaptive selector that chooses the best constitution per model.

By Yibo Yang, Zerui Huang, Haomiao Yang, Zixia Wang
llmsbenchmarks
More like this →
arXiv Computation and Language
Sep 30

Evaluating Alignment of Behavioral Dispositions in LLMs

arXiv:2602.11328v2 Announce Type: replace Abstract: As people turn to LLMs for social advice, understanding their behavior in such contexts becomes essential. In this work, we focus on behavioral dis...

By Amir Taubenfeld, Zorik Gekhman, Lior Nezry, Omri Feldman, Natalie Harris, Shashir Reddy, Romina Stella, Ariel Goldstein, Marian Croak, Yossi Matias, Amir Feder
llmssafety
More like this →
arXiv AI
Oct 19

On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management

The paper proposes a framework to evaluate the reliability of large language model (LLM) agents when they generate stakeholder preferences for multi‑criteria decision‑making in healthcare operations. It examines both explicit preferences—agents rating the importance of patient waiting time, provider workload, and operating cost in an outpatient scheduling scenario—and implicit preferences—agents’ pairwise choices between emergency department configurations. Experiments show that explicit preferences vary by role and operating conditions, but deliberation can homogenize them, while implicit preferences are heavily influenced by presentation order and only modestly improved by contextual models. Robust weighted‑sum optimization models that incorporate preference uncertainty often yield different schedules than those based on average weights, underscoring the need for auditing LLM‑generated preferences before using them in healthcare decision support.

By Marco Boresta, Tommaso Giovannelli, Sepehr Namirad, Giulia Riccardi
llmsagents
More like this →