arXiv AI July 21, 2026 By Zachary Wojtowicz, Ayush Nayak, Jacob Andreas

From Weights to Words: Expressing and Editing Preference Model Inferences in Natural Language

arXiv:2607. 16232v1 Announce Type: cross Abstract: The growing use of statistical learning algorithms to infer human preferences from high-dimensional choice data runs up against a fundamental challenge: choice alternatives typically differ in many ways simultaneously, so it is generally unclear which factors actually drove an observed decision and should be credited as preferences.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

llms

arXiv AI

Jun 2

Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning

arXiv:2606. 00334v1 Announce Type: cross Abstract: Various language domains have undergone remarkable changes in recent years; these shifts are largely attributed to the advent of Large Language Models and their misalignment with natural language usage.

By Xiaoyang Ming, Jose Hernandez, Thomas Stephan Juzek

llms reinforcement-learning safety

arXiv Computation and Language

Sep 10

Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge

arXiv:2602.02219v3 Announce Type: replace Abstract: Large language models are widely employed as evaluators, a paradigm commonly referred to as LLM-as-a-judge. Prior research has predominantly examin...

By Yuzheng Xu, Tosho Hirasawa, Tadashi Kozuno, Yoshitaka Ushiku

llms safety

arXiv AI

Jun 24

When Preferences Fail to Become Incentives: A Utility-Behavior Gap in Large Language Models

arXiv:2606. 22974v2 Announce Type: replace Abstract: Recent work on preference elicitation in large language models (LLMs) has demonstrated that, when given a series of choices between two outcomes, LLMs reveal a coherent, model-specific utility structure.

By Yujun Zhou, Christopher M. Ackerman

llms safety

arXiv AI

3d ago

Optimal Design for Active Preference Learning with Biased LLM Judges

arXiv:2609.38860v1 Announce Type: cross Abstract: Learning from human preferences is central to large language model (LLM) alignment, but human preference annotation is costly. Active preference lear...

By Zhongman Du, Huiming Zhang, Haodong Zhu, Baochang Zhang

llms safety

arXiv AI

4d ago

PADM\'E: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators

PADM'E is a method for synthesizing preference‑aligned data to meta‑evaluate language‑model (LM) evaluators of agentic behaviors. It reframes meta‑evaluation as a preference judgment problem, generating criterion‑based data with small LMs and no human involvement. In a prototype, PADM'E produced 1,000 samples across four domains and three criteria, and human validation showed agreement with human judgment rising from 73% to 85% compared to a naive baseline.

By Cheng Chang, Yining Mao, Peng Qi

llms agents safety

arXiv AI

Jun 9

A Regret Minimization Framework on Preference Learning in Large Language Models

arXiv:2606. 09124v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has enabled progress on reasoning-intensive tasks by relying on task-specific verifiers that provide automated correctness signals.

By Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim, Yu Jin Kim, Youngsoo Jang, Moontae Lee, Jungwoo Lee

llms reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning

Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge

When Preferences Fail to Become Incentives: A Utility-Behavior Gap in Large Language Models

Optimal Design for Active Preference Learning with Biased LLM Judges

PADM\'E: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators

A Regret Minimization Framework on Preference Learning in Large Language Models