UniPolicy is a multi-policy alignment framework for search advertising that jointly optimizes relevance, click propensity, and commercial value. It uses objective-specific prefix tokens, sparse MoE-LoRA routing, and residual FFNs to decouple parameters within a shared backbone, and builds pairwise preferences from multi-stage behavioral feedback to improve generation. In large-scale offline tests and a 7‑day online A/B test, UniPolicy achieved balanced gains across metrics, boosting CTR by 0.71%, RPS by 1.58%, and revenue by 1.32% while keeping latency stable.
arXiv:2512.23067v3 Announce Type: replace
Abstract: Personalized alignment from preference data has focused primarily on improving personal reward model (RM) accuracy, with the implicit assumption th...
By Fady Rezk, Yuangang Pan, Chuan-Sheng Foo, Xun Xu, Nancy Chen, Henry Gouk, Timothy Hospedales
arXiv:2609.08211v1 Announce Type: new
Abstract: Real-world Multi-Objective Reinforcement Learning (MORL) often suffers from sparse rewards, reward conflicts, and late-stage reward tug-of-war, causing...
By Shanwen Mao, Hao Zhang, Guangtao nie, Zhiheng Li, Huimu Wang, Sulong Xu, Gu Simiu
OneBid is a unified auto‑bidding foundation model that consolidates diverse cost‑per‑X (oCPX) advertising scenarios into a single framework. It builds on Decision Transformer by conditioning on two atomic signals—Return‑to‑Go for conversion value and Cost‑to‑Go for cost ratio—and incorporates value‑aware regularization. A sequence‑level Mixture‑of‑Experts architecture captures cross‑scenario knowledge while preserving low latency, and a Critic‑guided Relative Offline Policy optimization (CROP) aligns the backbone with scenario‑specific preferences without unsafe online exploration. In production at Kuaishou, OneBid achieved a 2.2% overall ADVV increase and up to 13.1% in the ROAS scenario.
By Yewen Li, Peng Jiang, Yitian Li, Pengfei Lv, Xialong Liu, Peng Jiang, Qingpeng Cai
The paper introduces GR4AD‑X, an enhanced generative recommendation framework for large‑scale advertising that tackles three key limitations of its predecessor: unreliable deterministic SID assignment, limited scalability of backbone models, and weak credit assignment in sequence‑level value optimization. GR4AD‑X improves SID reliability via Multi‑Path assignment, expands model capacity across complementary dimensions for better efficiency, and aligns generation with commercial value through Unified Value‑Guided Generation and Discrimination (UniVGD), which optimizes prefix‑level rewards, prunes beams, and reranks discriminatively. Online A/B tests on Kuaishou’s advertising platform show a 3.07% revenue increase over the baseline while meeting strict latency and throughput constraints.
By Yatong Xiao, Yibo Sheng, Lixiang Wang, Mingjie Sun, Yunhao Sha, Pengfei Zhang, Shengyu Wang, Peng Wang, Wenjin Wu, Peng Jiang
arXiv:2510. 01167v2 Announce Type: replace-cross Abstract: Aligning large language models to human preferences is inherently multidimensional, yet most pipelines collapse heterogeneous signals into a single objective.
By Yiran Shen, Yu Xia, Jonathan Chang, Prithviraj Ammanabrolu
arXiv:2607. 17281v1 Announce Type: cross Abstract: Auto-bidding plays an essential role in online advertising, automatically adjusting bids for advertisers to optimize their commercial goals.
By Yuejia Dou, Hesong Wang, Xinyu Zhang, Tianyu Wang, Zhilin Zhang, Chuan Yu, Jian Xu, Bo Zheng, Qi Qi
The paper introduces OGC-DPO, an active preference learning framework for aligning large language models with multiple objectives such as helpfulness, harmlessness, and honesty. OGC-DPO treats each response pair and objective as a separate query, using the model’s implicit gradient to gauge informativeness and selecting queries that expand gradient coverage for each objective. Experiments show that OGC-DPO outperforms existing methods at the same annotation cost and reveals that informative conflicts can serve as valuable training signals.
By Duc Anh Vu, Thong Nguyen, Hoang Pham, Ponhvoan Srey, Luu Anh Tuan
arXiv:2604. 05845v2 Announce Type: replace-cross Abstract: Auto-bidding services optimize real-time bidding strategies for advertisers under key performance indicator (KPI) constraints such as target return on investment and budget.
By Linghui Meng, Chun Gan, Shengsheng Niu, Chengcheng Zhang, Chenchen Li, Chuan Yang, Yi Mao, Xin Zhu, Jie He, Zhangang Lin, Ching Law
WeGR is a unified generative retrieval framework that tackles two key challenges in industrial search: preserving collaborative signals in tokenization and mitigating reward sparsity in reinforcement learning. It introduces the Collaborative and Collapse-free Tokenizer (CoCo‑VAE) to embed query‑to‑item and item‑to‑item signals into continuous representations before quantization, while also refreshing underutilized codewords to maintain balanced codebook usage. For preference alignment, Dynamic Horizon Policy Optimization (DHPO) expands reward evaluation horizons to enable coarse‑to‑fine learning from SID prefixes to full items without auxiliary reward models. Extensive offline tests and a six‑week online A/B test on a large‑scale e‑commerce platform demonstrate that WeGR improves retrieval and ranking performance, yielding relative gains of 2.539% in orders, 1.508% in CTR, and 1.381% in CVR, and reduces CPU and GPU usage by 25.5% and 26.1% respectively.
By Jiajun Han, Guoxin Ma, Felix, Liangsheng Xu, Ruiqi Yu, Yingdi Guo, Zongbing Wang, Xinhua Ye, Chenyan Hua, Xin Xie, Pengcheng Yang, Haijun Wu
arXiv:2610.34917v1 Announce Type: new
Abstract: Diffusion models enable multiple advertisers to compete for exposure within a shared generated image. Yet existing approaches are computationally costl...
By Yijie Su, Ruiyang Si, Tonghan Wang
arXiv:2608. 06792v1 Announce Type: cross Abstract: Foundation model(FM) for recommendation has shown strong ability to model long-horizon sequential user behavior.
By Oseong Choi, Hoeinn Kim, Jihoon Lee, Byungsoo Kang, Taeyeong Jang