arXiv AI
Reinforcement Learning with Conformal Action Sets: An Application to Sequential Recommendation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 12
Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target
The paper introduces Cross-Behavior Policy Optimization (CBPO), a method that unifies offline demonstrations and policy-generated rollouts into a single regression loss using a fixed-reference target. CBPO eliminates the need for separate supervised fine‑tuning and reinforcement learning objectives, allowing positive and negative offline completions, reference responses, and current and historical rollouts to share a common reward‑labeled format and replay buffer. Experiments on two model scales and four mathematical reasoning benchmarks show that CBPO, especially the CBPO‑Unified variant, outperforms baseline methods in avg@32 and pass@32 metrics, with historical rollouts providing additional gains.
arXiv Machine Learning
Jun 17
Learning Upper Lower Value Envelopes to Shape Online RL: A Principled Approach
arXiv:2510. 19528v2 Announce Type: replace-cross Abstract: We investigate the fundamental problem of leveraging offline data to accelerate online reinforcement learning - a direction with strong potential but limited theoretical grounding.
arXiv Machine Learning
Jun 2
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
arXiv:2606. 01123v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) avoids explicit reward engineering by learning from pairwise human preference feedback.
arXiv Machine Learning
Sep 29
From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning
The paper introduces Q-Target Pretrained Transformers (QTPT), a method that replaces supervised behavior cloning with a Bellman-style Q‑target objective for in‑context reinforcement learning. QTPT retains the context‑conditioned Transformer architecture but learns to estimate action values using rewards and transitions from the context, rather than merely imitating offline actions. The authors provide theoretical analysis in stochastic linear bandits and finite‑horizon MDPs, demonstrating improved robustness to weak or suboptimal data, and empirically show gains over supervised pretraining on controlled RL benchmarks and extensions to D4RL Kitchen and AntMaze.
arXiv Machine Learning
Sep 21
IncentRL: The Trade-Off Between Preference Guidance and Task Performance
arXiv:2609.21525v1 Announce Type: new Abstract: Preference-based reward shaping can guide reinforcement learning, but adding preference signals to the reward may unintentionally change the task being...
arXiv Machine Learning
Sep 30
Efficient Offline Learning of Ranking Policies via Top-$k$ Policy Decomposition
arXiv:2609.36740v1 Announce Type: new Abstract: Many recommender systems such as for e-commerce and news platforms aim to provide users with rankings they are likely to interact with. Off-Policy Lear...