Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2610.06647v2 Announce Type: replace Abstract: Reinforcement learning has greatly advanced the capabilities of large language models, but its memory demands remain a barrier to broader adoption....
arXiv:2610.10457v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) achieve remarkable performance in video synthesis, but their iterative denoising process suffers from high inference late...
The paper introduces an asynchronous value‑iteration RL algorithm for average‑reward multichain MDPs that requires only the transition graph and uses Bather’s decomposition to partition the state space into communicating subsystems and transient states. This hierarchical decomposition transforms the global decision problem into structured subproblems, enabling the algorithm to converge to the optimal gain and produce gain‑optimal policies in finite time. Two additional algorithms are presented: one that approximately solves the multichain average optimality equations for near gain‑optimality, and another that approximates the optimal bias function to achieve near bias‑optimality, both with almost‑sure convergence guarantees and improved transient performance. whyItMatters":"The work offers the first essentially model‑free average‑reward RL algorithms for general multichain MDPs, avoiding reductions to discounted problems."
arXiv:2604.13491v4 Announce Type: replace Abstract: With the rapid progress of Multimodal Large Language Models (MLLMs), unified MLLMs that jointly perform image understanding and generation have adv...
The paper studies how an adaptive attacker can evade machine‑learning intrusion detection systems (IDS) deployed on a Raspberry Pi 3B+ by using a Deep Q‑Network (DQN) to learn port‑scan tactics that vary probe timing, TCP flags, and payload size. Three IDS models—XGBoost, a multi‑layer perceptron, and a 1D convolutional neural network—trained on TON_IoT telemetry were evaluated under black‑box, gray‑box, and white‑box feature‑visibility scenarios. Evasion success rates ranged from 61.9 % to 98.3 %, showing that even limited feature knowledge can enable high‑rate evasion and that the impact of feature visibility depends on the specific IDS model.
arXiv:2610.09712v1 Announce Type: cross Abstract: We develop a continuous-time deterministic policy gradient framework for reinforcement learning with reflected state dynamics, where the state proces...
The paper investigates when allocation stability is required for Gaussian inference in a two‑armed, fixed‑horizon variance‑aware UCB policy with bounded rewards that may change with the horizon. It establishes a precise criterion involving the reward gap and variances that determines whether the optimal‑arm count can be approximated deterministically with vanishing relative error, while noting that the suboptimal‑arm count is always stable. The authors prove that the ordinary Wald statistic for any fixed nonzero coefficient vector converges to a standard normal distribution provided the product of pull count and reward variance diverges for each arm; however, uniform Gaussian approximation over deterministic coefficient vectors holds only if the optimal‑arm count is stable. The analysis relies on a pathwise comparison with an auxiliary policy and joint limits for the rescaled optimal‑arm count and studentized sample‑mean errors.
EmphTTS is a non‑autoregressive text‑to‑speech system that uses Group Relative Policy Optimization (GRPO) to train its duration predictor with an emphasis localization reward, allowing direct word‑level emphasis control. The system outperforms existing baselines in both objective emphasis evaluation and subjective preference tests, and ablation studies confirm that GRPO improves emphasis realization while reducing mismatch between the duration predictor and the TTS model.
arXiv:2610.09679v1 Announce Type: new Abstract: Adaptive rollout methods for group-relative reinforcement learning typically allocate a fixed per-update budget across prompts. We instead study how to...
arXiv:2610.10203v1 Announce Type: new Abstract: Model organisms of alignment-relevant behaviors (e.g., backdoors, sycophancy, spurious correlations) have emerged as a key tool for evaluating whitebox...
arXiv:2610.08970v1 Announce Type: cross Abstract: Humanoid loco-manipulation of large, heavy objects demands forceful interaction across the entire body. However, such payloads shift a humanoid's cen...
arXiv:2610.09294v1 Announce Type: cross Abstract: Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments. As agents move...
arXiv:2610.09384v1 Announce Type: cross Abstract: Language models are routinely fine-tuned under a fixed context, such as a generic system prompt, persona or domain-specific instruction, yet the lear...
arXiv:2610.09517v1 Announce Type: cross Abstract: Frozen vision-language models increasingly provide safety signals for reinforcement learning. Their use assumes that similarity to language describin...
arXiv:2610.09943v1 Announce Type: cross Abstract: Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the...
arXiv:2610.10206v1 Announce Type: cross Abstract: Links are disjoint unions of circles smoothly embedded in $S^3$. We use reinforcement learning and Bayesian optimisation to obtain new upper bounds o...
arXiv:2610.10407v1 Announce Type: cross Abstract: As option markets grow and AI advances, agentic systems for option trading are gaining increasing attention. Language-model-based agents can reason o...
arXiv:2610.06083v2 Announce Type: replace Abstract: Most adversarial attacks on deep reinforcement learning (DRL) assume white-box access to the victim policy, which rarely holds in practice. This pa...
arXiv:2605.19352v2 Announce Type: replace-cross Abstract: Understanding how humans and artificial intelligence systems predict and plan by interacting with their environment is a fundamental challeng...
arXiv:2610.09788v1 Announce Type: new Abstract: Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can in...