arXiv Machine Learning June 19, 2026 By Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari, Aryan Sajith, Hamed Zamani

Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users

Read the original on arXiv Machine Learning →

arXiv:2606. 20482v1 Announce Type: cross Abstract: To align a Large Language Model (LLM), most existing methods collect explicit human feedback and train a reward model to predict the human preference based on the response text.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 8

Re-Centering Humans in LLM Personalization

arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.

By Lechen Zhang, Jiarui Liu, Tal August

llms

arXiv AI

Jul 3

HAL: Inducing Human-likeness in LLMs with Alignment

arXiv:2601. 02813v3 Announce Type: replace Abstract: Aligning language models to qualitative behavioral traits, such as human-likeness, remains difficult because they are hard to define, measure, and optimize.

By Masum Hasan, Junjie Zhao, Ehsan Hoque

llms safety

arXiv AI

Aug 12

Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes

arXiv:2608. 10209v1 Announce Type: new Abstract: Feedback signals used to train Large Language Models (LLMs) are the primary driver of their behavior and our main lever for instilling alignment with human values and objectives.

By Alec Harris, Kasey Corra, Archie Chaudhury, Yixiong Hao

llms reinforcement-learning fine-tuning safety

arXiv AI

Jul 20

Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling

arXiv:2607. 15740v1 Announce Type: cross Abstract: As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI.

By Bo-An Chang, Yu-Chih Chen

llms diffusion nlp reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Jul 28

What do Reward Models Memorize?

arXiv:2607. 24484v1 Announce Type: new Abstract: This paper studies what discriminatively trained reward models (RMs) memorize by measuring counterfactual memorization on two human preference datasets.

By Ivo Verhoeven, Pushkar Mishra, Ekaterina Shutova

arXiv AI

Jul 7

Attention Limited Reward Learning

arXiv:2607. 04590v1 Announce Type: new Abstract: Pairwise human comparisons are a primary interface through which modern AI systems learn human preferences.

By Wenqian Xing

reinforcement-learning safety

Related stories

Re-Centering Humans in LLM Personalization

HAL: Inducing Human-likeness in LLMs with Alignment

Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes

Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling

What do Reward Models Memorize?

Attention Limited Reward Learning