← Back to all news
arXiv Machine Learning By Khurram Yamin, Jingjing Tang, Eric Horvitz, Bryan Wilder

Can Revealed Preferences Clarify LLM Alignment and Steering?

Read the original on arXiv Machine Learning →

The paper proposes an empirical pipeline to estimate the preferences that a large language model (LLM) implicitly optimizes by combining the model’s probability distribution over unknowns with its chosen action, and fitting a discrete choice model to recover the underlying cost function. This revealed-preference framework enables rigorous assessment of whether LLMs act consistently toward a goal, can articulate objectives that align with their decision policy, and can be steered by prompting to follow a user-specified cost function. Experiments across four medical diagnosis domains and various frontier and open-source models show that while many LLMs exhibit internal coherence, they still struggle to accurately report or adopt preferences when guided by users.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 10

When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMs

The paper introduces a decision‑theoretic framework that elicits both probability judgments and decisions from large language models (LLMs) to test whether their reported beliefs are consistent with their actions. It shows that this framework yields empirically testable conditions without assuming a specific utility function. In clinical diagnosis simulations, the authors find that while LLMs’ reported beliefs are not perfect reflections of the information in their decisions, the discrepancies are small for the strongest models.

By Khurram Yamin, Jingjing Tang, Santiago Cortes-Gomez, Amit Sharma, Eric Horvitz, Bryan Wilder
llmsagents
More like this →
arXiv AI
6d ago

Jointly Modeling Preferences and Imperfect Decision-Making in Large Language Models

The paper introduces the Preference-Behavior Model (PBM), which jointly infers Large Language Model (LLM) preferences and imperfect decision-making mechanisms from observed choices. By applying PBM to three social dilemma games—Public Goods, Collective-Risk, and Common-Pool Resource—the authors show that preferences derived from one dilemma can predict behavior in others, whereas traditional behavioral economic elicitation instruments fail to do so consistently. The study also evaluates how well inferred preferences align with persona instructions, revealing both broad agreement and specific divergences across dimensions, LLMs, and games.

By Axel Abels, Elias Fern\'andez Domingos, Tom Lenaerts
llmsagentssafety
More like this →
arXiv AI
5d ago

MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics

MedConstBench is a new medical‑ethics benchmark that treats a model’s deployment constitution as a controllable variable. It is organized around Beauchamp’s four principles and uses human‑calibrated peer review to score responses. The benchmark includes the MedDilemma dataset of clinical dilemmas and demonstrates that different constitutions influence LLM behavior in a dimension‑selective, model‑dependent way, leading to a proposed adaptive selector that chooses the best constitution per model.

By Yibo Yang, Zerui Huang, Haomiao Yang, Zixia Wang
llmsbenchmarks
More like this →