Hugging Face Trending Papers
Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalization
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Oct 16
Explaining the Roles of Experts in Mixture-of-Experts LLMs for Personalization
The paper investigates how to personalize Large Language Models by leveraging Mixture-of-Experts (MoE) architectures, which route tokens to a few specialized experts, thereby reducing polysemanticity compared to dense models. The authors adapt three open-weight MoE LLMs on a large personalization dataset, train crosscoders to learn interpretable personalized latent representations, and identify experts that encode distinct personalization features. Experiments show that personalized MoE LLMs match dense LLM performance while activating fewer parameters, exhibit lower polysemanticity, and that causal interventions on the identified experts can steer a non-personalized model toward personalized behavior.
arXiv AI
Jun 8
Re-Centering Humans in LLM Personalization
arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.
arXiv AI
Oct 7
MoF: Preference-Aware Mixture Modeling for Black-Box LLM Personalization
arXiv:2610.08330v1 Announce Type: new Abstract: Proprietary Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet aligning their outputs with divers...
arXiv AI
2d ago
Interpretability Is Not Just a Tax: Sparsity and Diversity Yield Legible and Composable Reward Experts for Personalization
The paper investigates how to make Mixture-of-Experts (MoE) reward models more interpretable and useful for personalization by focusing on sparsity and diversity regularizers. By keeping the architecture and training data constant, the authors show that conventional MoE reward models learn entangled components that hinder both interpretability and personalization. Introducing a sparse MoE objective reduces entanglement, yielding experts that align with semantic categories and functional strengths, and enables composable, few‑shot personalization with a 25.81‑point improvement over entangled models.
arXiv Computation and Language
Sep 10
HyperTrace: Hypothesis-Based Preference Tracing for Online LLM Personalization
HyperTrace is a training‑free framework that personalizes large language models by tracing latent user preferences online. It maintains interpretable natural‑language hypotheses about short‑term intent and long‑term preferences, updating them with an SMC‑style reweighting process driven by an LLM‑based surrogate choice model. Experiments on PRISM and PersonaMem‑v2 demonstrate that HyperTrace improves response alignment, preference prediction, and profile consistency compared to strong online baselines.
arXiv AI
Sep 7
PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspaces
PLUME is a lightweight framework for personalizing large language models by learning a shared task-specific subspace from aggregated user data and then training only a small square matrix for each user within that subspace. The approach introduces cross-layer shared parameters and rank‑1 residual terms to reduce redundancy while preserving expressiveness. Experiments on personalized text generation benchmarks show that PLUME matches or outperforms strong baselines while cutting per‑user parameters by over 95%.