arXiv AI August 20, 2026 By Emma Yanyang Kong, JJ Tan, Ishan Gupta, Lars Olds, Claire Campbell, David Fagnan, Veli Balin, Rohan Gosain, Louis Garcia, Minsu Jang

The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations

Read the original on arXiv AI →

The paper introduces a lifecycle framework for LLM-as-a-Judge systems used to evaluate recommendation explanations at Netflix. It outlines four phases—Birth, Training, Deployment, and Monitoring—detailing how each stage addresses specific technical and operational challenges. The authors report that after five weeks of A/B testing, judge-aligned explanations increased novel content viewing and successful browse-to-play sessions without quality takedowns.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 11

GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human

arXiv:2605. 28882v2 Announce Type: replace-cross Abstract: With the rapid advancement of large language models, evaluating human-likeness in open-ended conversation has become increasingly important.

By Yihang Lin, Yunze Gao, Zeyang Lin, Dongbo Li, Kun Peng, Yue Liu

llms agents benchmarks safety

arXiv AI

Jul 10

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry

arXiv:2601. 22588v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are widely used as reference-free evaluators via prompting, but this "LLM-as-a-Judge" paradigm is costly, opaque, and sensitive to prompt design.

By Zhuochun Li, Yong Zhang, Ming Li, Yuelyu Ji, Yiming Zeng, Ning Cheng, Yun Zhu, Yanmeng Wang, Shaojun Wang, Jing Xiao, Daqing He

llms benchmarks

arXiv AI

Jul 24

Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents

arXiv:2602. 10226v2 Announce Type: replace-cross Abstract: Optimizing large-scale machine learning systems, such as recommendation models for global video platforms, requires navigating a massive hyperparameter search space and, more critically, designing sophisticated optimizers, architectures, and reward functions to capture nuanced user behaviors.

By Haochen Wang, Yi Wu, Daryl Chang, Li Wei, Lukasz Heldt

llms agents efficiency

arXiv AI

Jul 23

Co-Evolving LLM Evaluators and Policies via DynamicRubric

arXiv:2607. 20083v1 Announce Type: cross Abstract: Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models.

By Beining Wang, Weihang Su, Hongtao Tian, Hao Kong, Tao Yang, Ting Yao, Qingyi Pan, Yueyue Wu, Qingyao Ai, Min Zhang, Yiqun Liu

llms reinforcement-learning

arXiv AI

Aug 11

LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing

arXiv:2608. 08889v1 Announce Type: new Abstract: Recommendation systems thrive on personalization, where ''correctness'' is rarely a binary truth but a matter of subjective human preference.

By Juncheng Dong, Ding Tong, Ishan Gupta, Yuyan Wang

llms agents reinforcement-learning safety

arXiv AI

Jul 13

All Explanations are Wrong, But Many Are Useful: Exploring the Rashomon Explanation Set with Large Language Models

arXiv:2607. 09502v1 Announce Type: cross Abstract: Explaining machine-learning models is increasingly important for decision-making and consumer trust, yet it is widely believed to come at a cost: existing Explainable AI (XAI) methods suffer from a persistent accuracy-explainability trade-off.

By Pan Li

llms agents benchmarks

One email a morning, machine-written

Related stories

GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human

Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry

Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents

Co-Evolving LLM Evaluators and Policies via DynamicRubric

LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing

All Explanations are Wrong, But Many Are Useful: Exploring the Rashomon Explanation Set with Large Language Models