Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2607.20372v2 Announce Type: replace Abstract: Humans distill experience into reusable abstractions, e.g., strategies and cautionary reminders, and apply them to gradually solve problems more ef...
arXiv:2605.31378v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) still struggle with fine-grained translation quality estimation (QE), even with long reasoning chains. We argue that...
arXiv:2602.13551v3 Announce Type: replace Abstract: Reward models (RMs) play a central role throughout the language model (LM) pipeline, particularly in non-verifiable domains. However, the dominant...
The paper introduces Personalized Group Relative Policy Optimization (P‑GRPO), a new alignment framework for large language models that separates advantage estimation from immediate batch statistics. By normalizing advantages using preference‑group‑specific reward histories instead of the concurrent generation group, P‑GRPO maintains contrastive signals for distinct user preferences. Experiments across various tasks show that P‑GRPO converges faster and yields higher rewards than standard GRPO, improving alignment with heterogeneous human preferences while preserving general capabilities.
arXiv:2608.29378v1 Announce Type: cross Abstract: Arabic large language models must refuse harmful prompts without over-refusing benign or sensitive prompts, yet a single refusal rate hides this trad...
arXiv:2605.29089v2 Announce Type: replace-cross Abstract: Recent reinforcement learning (RL) post-training approaches primarily optimize the final output policy using sparse outcome-level rewards, wh...
arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...
arXiv:2502.06584v2 Announce Type: replace Abstract: Early Classification of Time Series (ECTS) is vital in fields like industrial monitoring and medical triage, where quick and accurate predictions a...
arXiv:2503.00030v3 Announce Type: replace-cross Abstract: Self-play-based policy optimization has emerged as an effective approach for fine-tuning large language models (LLMs), formulating preference...
RAGDiffusion++ advances garment generation by addressing the high‑frequency texture gap that previous retrieval‑augmented models left unresolved. The approach introduces a dual‑image FLUX architecture trained on a large, complex garment dataset, coupled with a new attribute‑aware reward model that guides reinforcement learning to favor realistic high‑frequency patterns. An adversarial‑regularized RL strategy (AR‑GRPO) further prevents artifact exploitation, ensuring the model samples authentic, detailed garment textures.
Agent2UCB is a new agentic system designed for Generative Engine Optimization (GEO), which refines content to boost its likelihood of being cited or summarized by generative AI search engines. The system autonomously evaluates nine GEO strategies for each content item, selects the most effective one, and speeds up this selection using a bandit-based Agent2UCB policy that blends large language model priors with real-time reward signals. Additionally, it offers a lightweight, text-only SEO readiness check that assesses readability, topical coverage, and EEAT-style credibility, and experiments on GEO-Bench demonstrate consistent visibility gains while maintaining SEO quality.
arXiv:2608.30399v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit strong semantic reasoning and open-ended generation abilities, but aligning these abilities with structured sequ...
arXiv:2506.17007v3 Announce Type: replace Abstract: A major bottleneck in scientific discovery consists of narrowing an exponentially large set of objects, such as proteins or molecules, to a small s...
arXiv:2608.29953v1 Announce Type: new Abstract: Flat retrieval-augmented generation treats a corpus as a bag of chunks, discarding document hierarchy and cross document structure. We introduce Search...
arXiv:2310.02779v3 Announce Type: replace Abstract: Generative flow networks (GFlowNets) are sequential sampling models trained to match a given distribution. GFlowNets have been successfully applied...
Normalized Low-Rank Adaptation (NoRA) is a lightweight enhancement to the widely used LoRA technique that normalizes the down‑projection matrices during training. By doing so, NoRA stabilizes early optimization dynamics, accelerates convergence, and improves performance across pretraining, supervised fine‑tuning, and reinforcement learning. The method adds no extra trainable parameters or inference‑time cost, making it broadly applicable.
arXiv:2608.29897v1 Announce Type: new Abstract: Long-horizon agents need a context manager to compress growing interaction histories into a bounded working context, via passive strategies or active s...
The paper reports on training autonomous driving policies via self‑play, extending previous work by using Transformers and a real‑city high‑definition map. On CARLA and Waymo benchmarks, the resulting policies underperform compared to Gigaflow, with identified failure modes such as reward hacking at traffic lights and lack of incentive to stop at stop signs. The authors also analyze which traffic rules emerge from self‑play and confirm that reward conditioning produces diverse driving behaviors.
arXiv:2608.30632v1 Announce Type: cross Abstract: Reinforcement learning (RL), particularly RL with Verifiable Rewards (RLVR), has recently emerged as a central paradigm for enhancing large language...
arXiv:2210.00580v4 Announce Type: replace Abstract: This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically used to m...