arXiv Machine Learning

Drift-Aware Multimodal User Representation Learning via Multi-Scale Temporal Modeling and Sparse Mixture-of-Experts

The paper introduces DUMoE, a drift‑aware multimodal user representation framework that models user preferences over time by integrating static profiles, short‑term signals, and long‑term dependencies. It employs a sparse mixture‑of‑experts interest adapter, where each expert captures a distinct latent interest and a gating network selects relevant experts for each user. A three‑stage training strategy decouples backbone learning, expert specialization, and gating optimization, and experiments on real social media data demonstrate that DUMoE outperforms existing methods in user interest and interaction prediction.

arXiv Machine Learning
Aug 3

GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System

arXiv:2607. 29213v1 Announce Type: cross Abstract: Modern recommender systems in food delivery increasingly leverage multimodal signals, including images, text, and user interaction histories, to enhance user experience, yet effective fusion of these heterogeneous modalities remains challenging, hindering both the joint modeling of multimodal signals and adaptation to evolving user intent.

By Jiping Liu, Zhongmin Zhang, Zisen Sang, Zhijia Fang, Tao Ouyang, Ma Jiang, Shaopeng Liang, Zeyang Hou, Guodong Cao, Jia Jia
arXiv AI
Aug 19

M3TR: Temporal Retrieval Enhanced Multi-Modal Micro-video Popularity Prediction

M3TR is a temporal retrieval‑enhanced multi‑modal framework for predicting micro‑video popularity. It introduces a Mamba‑Hawkes Process module to model user feedback as self‑exciting events, capturing long‑range temporal dependencies. A temporal‑aware retrieval engine then identifies historically relevant videos by combining multi‑modal content similarity with popularity trajectory similarity, augmenting the target video’s features for improved prediction accuracy.

By Jiacheng Lu, Weijian Wang, Mingyuan Xiao, Yang Hua, Tao Song, Bo Peng, Cheng Hua, Haibing Guan
arXiv AI
Jun 4

Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling

arXiv:2606. 04284v1 Announce Type: cross Abstract: Preference modeling plays a central role in reinforcement learning from human feedback (RLHF), enabling large language models (LLMs) to align with human values.

By Yifan Wang, Jinyi Mu, Mayank Jobanputra, Yu Wang, Ji-Ung Lee, Soyoung Oh, Isabel Valera, Vera Demberg
arXiv Machine Learning
Aug 17

MedMix: Specialization-Consistent Federated Sparse MoEs under Modality Heterogeneity

arXiv:2608. 13911v1 Announce Type: new Abstract: Federated multimodal medical AI faces modality heterogeneity at both the client and sample levels: clients may systematically lack access to specific modality types, while individual records within the same client may contain different partial modality subsets.

By Adiba Orzikulova, Dong Min Kim, Jaehong Yoon, Sung-Ju Lee
arXiv Machine Learning
Sep 7

MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

MURAL is a multimodal recommendation framework that replaces static similarity graphs with a dynamic topology discovery process. It uses an Adaptive Edge Learner to find latent item-item correlations efficiently and an Uncertainty-Aware Fusion module to down‑weight noisy modality signals based on aleatoric uncertainty. The model also incorporates a contrastive teacher‑student alignment to stabilize training and has been shown to outperform state‑of‑the‑art baselines on large‑scale TikTok and Amazon datasets, providing both higher accuracy and interpretability.

By Ahmad Mousavi (Department of Mathematics,Statistics American University), Majid Alikhani (Independent Researcher), Yeon-Chang Lee (Department of Computer Science,Engineering Ulsan National Institute of Science,Technology), Roberto Corizzo (Department of Computer Science American University), Yeganeh Abdollahinejad (Department of Biosystems,Agricultural Engineering Michigan State University)