arXiv AI

Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix

The paper introduces a counterfactual observability framework for Netflix’s recommender systems, aiming to disentangle raw engagement signals—such as views and clicks—from confounding factors like content quality, model behavior, presentation bias, and audience reach. It proposes three stakeholder‑centered principles and measurement methods that reduce bias, assess relativity, and capture incrementality, applicable to both single‑stage and cascading recommender architectures. The framework is demonstrated through multiple production deployments, showing its effectiveness in enhancing observability across Netflix’s recommendation pipelines.

arXiv Machine Learning
Jun 9

The Value of Personalized Recommendations: Evidence from Netflix

arXiv:2511. 07280v5 Announce Type: replace-cross Abstract: Personalized recommendation systems shape much of user choice online, yet their targeted nature makes separating out the value of recommendation and the underlying goods challenging.

By Kevin Zielnicki, Guy Aridor, Aur\'elien Bibaut, Allen Tran, Winston Chou, Nathan Kallus
Hugging Face Trending Papers
Sep 17

Reproducing Transparent and Scrutable Recommendations: Exploring Open-Weight Models via Natural-Language User Profiles

The study reproduces a prior work on recommender systems that use generated natural‑language user profiles to enhance transparency and user control. It confirms that the User Profile Recommendation (UPR) model performs competitively and that altering these profiles uniformly shifts predicted ratings without changing ranking order. Additional experiments include context ablation, multi‑seed stability, and mechanistic interpretability analysis with the nnsight framework.

arXiv AI
Sep 18

Reproducing Transparent and Scrutable Recommendations: Exploring Open-Weight Models via Natural-Language User Profiles

This reproducibility study confirms that incorporating generated natural‑language user profiles into recommender systems enhances transparency and allows users to directly intervene by correcting preferences or addressing cold‑start issues. The authors replicated the original findings and extended the evaluation with context ablation, multi‑seed stability tests, and mechanistic interpretability analysis using nnsight. Their results show that while perturbing profiles shifts predicted ratings uniformly across genres, the overall rankings remain unchanged, attributing this to the rating‑regression objective rather than the profile interface.

By Noah Mami\'e, Laurin van den Bergh
arXiv AI
Oct 28

Counterfactual Profile Perturbations for Recommendation Explanations: A Reproducibility Study

This reproducibility study investigates the counterfactual explanation method Learning to eXplain Recommendations (LXR) by extending its application beyond the top‑ranked item to other recommended items. The authors compare LXR against alternative baselines such as item popularity and latent factor similarity, noting that these baselines perform surprisingly well on certain tasks. They also analyze how item popularity affects explanation quality, finding that LXR produces shorter perturbations for less popular items, and conduct a brute‑force search to estimate a lower bound on optimal perturbation size.

By Meysam Varasteh, Elizabeth McKinnie, Robin Burke
arXiv AI
Sep 15

Safety as a Constraint: Fine-Tuning a LLM Recommender to Explain Itself

The paper presents a method for fine‑tuning a large language model (LLM) recommender to generate personalized, non‑harmful explanations for its recommendations. By training two LLM‑judge reward models and using constrained GRPO, the authors achieve a significant increase in the PASS rate for all three criteria, from 0.649 to 0.956 on their own judges and from 0.677 to 0.931 on an independent judge. The fine‑tuned model maintains its original recommendation performance, demonstrating that LLM‑based recommenders can be adapted to complex tasks without loss of effectiveness.

By Jiashu He, Emma Yanyang Kong, JJ Tan, David Fagnan
arXiv AI
6d ago

From Counterfactual Edits to Human Narratives: LLM-Driven Interpretations for Recommender System Explanations

The paper explores how large language models can transform extensive counterfactual edit lists from recommender systems into concise, natural‑language explanations that highlight higher‑level preference patterns. By optimizing these mediators for correctness, informativeness, and linguistic quality, the authors demonstrate that the resulting summaries maintain high linguistic quality and improve semantic coherence, especially when paired with effective counterfactual generators like ACCENT and SPINRec. Human‑calibrated metrics confirm that preference optimization enhances the overall quality of the explanations across multiple settings.

By Vojtech Cahlik, Amir Reza Mohammadi, Rodrigo Alves, Pavel Kordik, Eva Zangerle
arXiv AI
Sep 10

Neutralizing Popularity Bias in LLM-based Recommendation via Counterfactual Reasoning Guidelines

The paper introduces NPRec, a model‑agnostic framework that uses counterfactual reasoning to neutralize popularity bias in large language model–based recommender systems. By generating debiased textual guidelines that separate intrinsic user interests from popularity signals, NPRec injects these guidelines at inference time to guide the LLM’s generation without updating parameters. Experiments on three real‑world datasets show improved recommendation accuracy, explanation quality, and debiasing performance.

By Guanrong Li, Haolin Yang, Xinyu Liu, Zhen Wu, Rui Xia, Xinyu Dai