Can Revealed Preferences Clarify LLM Alignment and Steering?

The paper proposes an empirical pipeline to estimate the preferences that a large language model (LLM) implicitly optimizes by combining the model’s probability distribution over unknowns with its chosen action, and fitting a discrete choice model to recover the underlying cost function. This revealed-preference framework enables rigorous assessment of whether LLMs act consistently toward a goal, can articulate objectives that align with their decision policy, and can be steered by prompting to follow a user-specified cost function. Experiments across four medical diagnosis domains and various frontier and open-source models show that while many LLMs exhibit internal coherence, they still struggle to accurately report or adopt preferences when guided by users.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• safety