Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2608.30765v1 Announce Type: new Abstract: Multi-task reinforcement learning (MTRL) is a technique to train multiple tasks simultaneously, where previous works usually train a single model to so...
arXiv:2310.04363v3 Announce Type: replace Abstract: Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits t...
arXiv:2410.01432v3 Announce Type: replace Abstract: Amortized inference is the task of training a parametric model, such as a neural network, to approximate a distribution with a given unnormalized d...
arXiv:2512.23927v3 Announce Type: replace-cross Abstract: Fitted \(Q\)-iteration (FQI) is a standard regression-based method for optimal control in offline reinforcement learning, but its stability u...
arXiv:2605.31378v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) still struggle with fine-grained translation quality estimation (QE), even with long reasoning chains. We argue that...
arXiv:2601.03468v2 Announce Type: replace Abstract: Reinforcement learning (RL) has become a standard approach for post-training large language models and, more recently, for improving image generati...
arXiv:2608.29030v1 Announce Type: new Abstract: In-context watermarking (ICW) prepends an instruction to a query asking the model to embed a statistically detectable signal in its response. It thus e...
arXiv:2606.22394v3 Announce Type: replace Abstract: Consistency distillation has significantly accelerated diffusion-model inference, but its sampling dynamics remain underexplored. We reveal an asym...
The paper investigates how different forms of compressed chain‑of‑thought (CoT) reasoning—Explicit, Composed, and Implicit—affect large language model (LLM) performance after supervised fine‑tuning (SFT). Using a synthetic compositional reasoning task, the authors show that coarser CoT requires more SFT data, that Composed and Implicit CoT benefit more from data scaling (with Composed also benefiting from repetition), and that reinforcement learning with verifiable rewards (RLVR) can decompose compressed steps learned during SFT. Additionally, unidirectional CoT ordering improves generalization on longer sequential tasks.
The paper introduces Common-Trace Factorized Constrained Proximal Policy Optimization (CT‑PPO) for optimizing online sensing‑session consolidation in multi‑tenant integrated sensing and communication (ISAC) networks. By formulating the problem as a constrained Markov decision process, CT‑PPO shares workload traces across stochastic policy replicas and uses leave‑one‑out discounted Monte Carlo returns to credit applicable actor factors, achieving higher macro returns and lower sensing‑resource costs than baseline methods. Ablation studies show that the common‑trace reward credit is the key contributor to performance gains, and the algorithm retains advantages across varying arrival loads without increasing deployment latency.
arXiv:2608.28859v1 Announce Type: cross Abstract: Reasoning models do not stop when they know the answer. On DeepSeek-R1-Distill-Qwen-7B the chain of thought runs about twice as long as the model's o...
arXiv:2608.29296v1 Announce Type: cross Abstract: Larger batches reduce the variance of stochastic gradients per update and are therefore often expected to accelerate training. Yet whether this stati...
arXiv:2608.29458v1 Announce Type: cross Abstract: Sandbagging, in which a model deliberately underperforms on an evaluation despite retaining the underlying capability, threatens the safety evaluatio...
arXiv:2608.30122v1 Announce Type: cross Abstract: Vision-language-action (VLA) driving methods increasingly combine multi-trajectory imitation learning with group-relative policy optimization (GRPO),...
arXiv:2608.30935v1 Announce Type: cross Abstract: Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embod...
arXiv:2512.03244v2 Announce Type: replace-cross Abstract: Training process reward models (PRMs) requires step-level correctness labels, obtained either through expensive human annotation or by relyin...
arXiv:2602.08136v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are now a core part of modern AI. Recent work proposed several visual jailbreak attacks using single/ holistic...
arXiv:2605.29888v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training has shown to improve reasoning in large language models (LLMs). However, there has been little expl...
The paper reports on training autonomous driving policies via self‑play, extending previous work by using Transformers and a real‑city high‑definition map. On CARLA and Waymo benchmarks, the resulting policies underperform compared to Gigaflow, with identified failure modes such as reward hacking at traffic lights and lack of incentive to stop at stop signs. The authors also analyze which traffic rules emerge from self‑play and confirm that reward conditioning produces diverse driving behaviors.
arXiv:2506.17007v3 Announce Type: replace Abstract: A major bottleneck in scientific discovery consists of narrowing an exponentially large set of objects, such as proteins or molecules, to a small s...