arXiv AI

Group Preference Collapse in Personalized Multimodal Large Language Models

arXiv:2607. 22603v1 Announce Type: new Abstract: Personalized multimodal large language models (MLLMs) aim to generate user-specific responses, but existing methods mainly rely on profile-level information and overlook diverse user preferences.

Hugging Face Trending Papers

Jun 25

SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context

Personalized language-model assistants are often evaluated through a memory lens: can a model recall preferences users have explicitly stated in dialogue? More comprehensive personalization demands a harder capability -- inferring what users care about from the multimodal traces they naturally leave behind.

llms multimodal benchmarks

arXiv Machine Learning

Jun 30

Synthetic Interaction Data for Scalable Personalization in Large Language Models

arXiv:2602. 12394v2 Announce Type: replace Abstract: Personalized prompting offers large opportunities for deploying large language models (LLMs) to diverse users, yet existing prompt optimization methods primarily focus on task-level optimization while largely overlooking user-specific preferences and latent constraints of individual users.

By Yuchen Ma, Yue Huang, Wenjie Wang, Xiaonan Luo, Xiangliang Zhang, Stefan Feuerriegel

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 4

Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling

arXiv:2606. 04284v1 Announce Type: cross Abstract: Preference modeling plays a central role in reinforcement learning from human feedback (RLHF), enabling large language models (LLMs) to align with human values.

By Yifan Wang, Jinyi Mu, Mayank Jobanputra, Yu Wang, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg

llms reinforcement-learning safety

arXiv AI

Jun 9

A Dataset for Dynamic Human Preferences for Vision Language Models

arXiv:2606. 07653v1 Announce Type: cross Abstract: Given the increased adoption of Vision Language Models (VLMs) in human-interactive settings, it is important that we evaluate how well these models can adapt to real-time preferences for different users.

By Hannah Gao (Massachusetts Institute of Technology), Dylan Hadfield-Menell (Massachusetts Institute of Technology), Rachel Ma (Massachusetts Institute of Technology)

llms multimodal benchmarks

arXiv AI

Jun 9

ZIPP:Zero-shot Image Personalization from Personas

arXiv:2606. 08841v1 Announce Type: new Abstract: Text-to-image diffusion models are increasingly deployed in open-ended creative contexts, yet their outputs remain impersonal, optimized for aggregate aesthetics rather than individual taste.

By Harini SI, Somesh Singh, Yaman Kumar Singla, David Doermann, Rajiv Ratn Shah

llms diffusion fine-tuning benchmarks safety

arXiv AI

Aug 6

GeoReward: Mitigating Contextual Variable Overestimation in Vision-Language Models for Cross-Market Preference Prediction

arXiv:2608. 04504v1 Announce Type: cross Abstract: Vision-language models excel in many multimodal tasks but remain prone to a subtle yet impactful failure mode: they tend to overestimate dominant visual-textual cues while underestimating sparse but decision-critical contextual variables.

By Shuo Liu, Huixiang Cai, Weiru Zhang, Xiaoyi Zeng

llms diffusion reinforcement-learning fine-tuning multimodal safety

arXiv Machine Learning

Aug 3

GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System

arXiv:2607. 29213v1 Announce Type: cross Abstract: Modern recommender systems in food delivery increasingly leverage multimodal signals, including images, text, and user interaction histories, to enhance user experience, yet effective fusion of these heterogeneous modalities remains challenging, hindering both the joint modeling of multimodal signals and adaptation to evolving user intent.

By Jiping Liu, Zhongmin Zhang, Zisen Sang, Zhijia Fang, Tao Ouyang, Ma Jiang, Shaopeng Liang, Zeyang Hou, Guodong Cao, Jia Jia

rag reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

5d ago

Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization

arXiv:2608. 12389v1 Announce Type: new Abstract: Cross-domain zero- or few-shot personalization aims to generate user-preferred responses in unseen conversational domains from only a handful of target-domain interactions.

By Xuefei Wang, Jun Han, Zixuan Wang, Qingkai Zeng, Xiao Wang, Ruijie Wang, Jianxin Li

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 14

MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search

arXiv:2607. 11030v1 Announce Type: cross Abstract: Multimodal information is pivotal for e-commerce search ranking.

By Zhen-Lin Chen, Maosen Sheng, Peng Lin, Jianmin Chen, Zhuojian Xiao, Dongyue Wang, Xiwei Zhao

llms fine-tuning multimodal

arXiv Machine Learning

Jun 5

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding

arXiv:2505. 05026v5 Announce Type: replace-cross Abstract: User interface (UI) design goes beyond visuals to shape user experience (UX), underscoring the shift toward UI/UX as a unified concept.

By Jaehyun Jeon, Min Soo Kim, Jang Han Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Dae Hyun Kim, Youngjae Yu

llms multimodal benchmarks safety

arXiv AI

Jun 2

Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition

arXiv:2606. 00959v1 Announce Type: new Abstract: Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment.

By Wanlong Fang, Tianle Zhang, Wen Tao, Alvin Chan

llms multimodal benchmarks safety

arXiv AI

Aug 11

Learning Preference Adaptation for Large Language Model Personalization via Verbal Reinforcement Learning

arXiv:2608. 09507v1 Announce Type: cross Abstract: Natural language user preferences provide an interpretable interface for LLM personalization.

By Yuting Liu, Wei Wu, Jianzhe Zhao, Guibing Guo

llms rag agents reinforcement-learning

Related stories