Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2609.06667v1 Announce Type: new Abstract: Adaptive dosing requires policies that reduce tumor burden without excessive toxicity. Learned dosing policies are typically judged against historical...
arXiv:2403.05006v2 Announce Type: replace-cross Abstract: Pluralistic alignment requires learning from feedback that reflects persistent and potentially conflicting stakeholder preferences while ulti...
arXiv:2609.07137v1 Announce Type: cross Abstract: Recent image-to-3D generation models built on flow-matching diffusion Transformers (DiT) can produce high-fidelity meshes, yet their post-training st...
arXiv:2609.07461v1 Announce Type: new Abstract: We consider reinforcement learning in environments with dynamics that undergo an irreversible phase transition governed by a hidden temporal pattern. T...
arXiv:2609.05800v1 Announce Type: new Abstract: Activation steering controls LLM behavior at inference time by adding learned directions to hidden states, but existing methods handle one concept at a...
arXiv:2601.09233v3 Announce Type: replace-cross Abstract: The prevailing post-training paradigm for Large Reasoning Models (LRMs)---Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL...
arXiv:2601.12053v2 Announce Type: replace-cross Abstract: While foundation models have achieved remarkable results across a diversity of domains, they still rely on human-generated data, such as text...
arXiv:2609.09981v1 Announce Type: new Abstract: We study offline inference for the optimal value in reinforcement learning. Two new nuisances are derived as fixed points of a self-induced Bellman equ...
The paper introduces CoRL, a co-evolutionary reinforcement learning framework designed to defend against adaptive indirect prompt-injection attacks on tool-augmented language agents. CoRL operates in three stages—attacker fine‑tuning, bilateral Co‑PPO training, and defender fine‑tuning—using verifier‑grounded repairs to adapt to changing attack strategies. Experiments on 1,514 executions show that CoRL reduces attack success rates to 0% while improving task utility, demonstrating its effectiveness against adaptive adversaries.
arXiv:2511.05933v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) is often credited with improving reasoning at the expense of factual knowledge. We instead find that reasoning mo...
arXiv:2609.05861v1 Announce Type: cross Abstract: One important AI-for-RAN use case is energy saving, in which radio resources and cell energy modes must be dynamically controlled without violating u...
arXiv:2605.30273v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empat...
arXiv:2609.05650v1 Announce Type: new Abstract: We propose a reinforcement learning framework in which exploration is driven by intrinsic curiosity, designed for scenarios where environments are non-...
arXiv:2609.06647v1 Announce Type: new Abstract: Although professional workflows leverage large language models widely, the interpretation for auditing unconstrained free-text generation is usually in...
arXiv:2609.06289v1 Announce Type: cross Abstract: As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inferenc...
The paper introduces Feedback‑Enriched Environments (FEEs) as a new approach to training large language models as autonomous agents for long‑horizon tasks. By shifting from action guidance to observation enrichment during later stages of exploration, FEEs improve performance across SciWorld and BFCL benchmarks with various Qwen3 model scales and RL algorithms. The study shows that FEEs stabilize training, promote proactive exploration, embed environmental guidance into policy weights, and highlight intra‑group feedback consistency as key for stable optimization.
HyCO is a hybrid neural solver that combines a sequential reinforcement learning (RL) solver with a global diffusion model (DM) to tackle combinatorial optimization problems. The RL component builds an initial solution prefix, after which HyCO switches to a conditional DM to finish the remaining decisions. The authors provide a theoretical framework showing that this hybrid approach yields lower expected regret than either method alone, identify an optimal trigger step for the switch, and implement a lightweight adaptive trigger based on policy entropy and RL‑DM disagreement, achieving consistent performance gains across benchmarks.
The study shows that instruction‑tuned language models, when asked to generate responses with varying emotional intensity, consistently undershoot the requested affect. By conditioning a model on continuous Valence‑Arousal targets and measuring output with a frozen regressor, the authors find that the gain for valence is only 0.26 and for arousal 0.13 on Llama‑3.1‑8B, far below the ideal value of 1. They trace this undershoot to the preference‑learning pipeline: training data such as EmoBank are neutral‑heavy and the candidate pool rarely contains extreme affect, so Direct Preference Optimization lacks examples to learn from. Expanding the target space uniformly and sampling a hotter, larger candidate pool raises valence gain to 0.40 and improves extrapolation with minimal in‑distribution cost, a result that also holds for Qwen3‑8B. Arousal remains more variable because the base model rarely generates highly aroused candidates.
The paper introduces Long-Short Term Advantage Estimator (LSTAE), a single‑stream reinforcement learning algorithm that replaces repeated within‑iteration trajectory sampling with historical experience for advantage estimation. LSTAE tracks each task anchor with a drift‑aware long‑term baseline and a short‑term state‑experience buffer, converting accumulated experience into multi‑granular credit signals while requiring only one rollout per anchor. Experiments on agentic and mathematical reasoning benchmarks show that LSTAE matches or surpasses strong group‑based baselines while significantly reducing rollout cost.
The paper introduces a structured post‑training pruning method for vision and language transformers called Damage‑Aware Bandit Pruning. It treats the selection of functional units (attention heads and MLP channel groups) as a multi‑armed bandit problem, using paired damage (masked loss minus base loss) as a reward to guide either UCB or Thompson Sampling policies. Experiments on a range of models (GPT‑2, OPT, Pythia, Qwen2.5, SmolLM2, ViT‑B/16, DeiT‑Tiny, Swin‑Tiny) show that the bandit approaches generally reduce degradation compared to budgeted‑greedy baselines, with statistically significant improvements in most comparisons.