Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2609.27717v1 Announce Type: new Abstract: Human-written agent skills encode rich workflows for real-world problem solving, but are typically used as external inference-time instructions rather...
arXiv:2609.27455v1 Announce Type: new Abstract: World Action Models (WAMs) jointly model action generation and environment dynamics and are mostly built on pretrained Video Diffusion Models (VDMs). I...
arXiv:2609.27667v1 Announce Type: new Abstract: Reinforcement learning (RL) achieves strong performance in sequential decision-making but remains brittle under dynamic uncertainty and distributional...
arXiv:2609.27051v1 Announce Type: new Abstract: Language-model agents now run the whole of quantitative factor research: they propose investment factors, backtest them, select the survivors and retir...
arXiv:2609.27284v1 Announce Type: new Abstract: We present Hunyuan-A13B, an open-source large language model based on a Mixture-of-Experts architecture. It contains 80 billion total parameters but ac...
arXiv:2609.27450v1 Announce Type: cross Abstract: Vision-language-action (VLA) models handle long-horizon manipulation, yet success hinges on a few precision-critical phases where millimeter-scale er...
arXiv:2609.27572v1 Announce Type: cross Abstract: Reinforcement learning (RL) has emerged as a key paradigm for improving the reasoning capabilities of large language models (LLMs). However, existing...
arXiv:2609.27421v1 Announce Type: new Abstract: Multi-constraint instruction following requires a model to respond to a query under many simultaneously active constraints. Even strong instruction-tun...
arXiv:2609.27532v1 Announce Type: new Abstract: Long-horizon agentic tasks require an agent to modify an environment through a sequence of tool calls, with success determined by the final state. The...
The paper investigates what a steerable valence direction in large language models (LLMs) actually represents, focusing on a good‑bad outcome direction in a maze task. By using controlled interventions that separate the realized outcome from the informational history that led to it, the authors find that directions trained on one explicit outcome encoding transfer well to another, suggesting the readout is not tied to surface form. However, when the same outcome is achieved through announced versus unannounced histories, transfer performance drops sharply, indicating that the post‑event readout remains strongly conditioned on the earlier announcement. In a matched maze‑reinforcement‑learning run, the post‑RL direction becomes more predictive of reference‑MDP return and the policy depends more on it, yet the history dependence persists. These findings support a functional, value‑related interpretation of the direction but argue against identifying it with a history‑invariant scalar valence state.
Group-based reinforcement learning (RL) methods, such as GRPO and its variants, have become a leading paradigm for training reasoning and agentic large language models (LLMs). While their group-normal...
In value-based reinforcement learning, improving the accuracy of policy evaluation has been shown to improve downstream policy optimization performance. The widely adopted family of approximations rel...
Reinforcement learning (RL) is widely used to sharpen reasoning in multimodal large language models (MLLMs), yet its effect on hallucination is uneven. We trace this to two weak points in the \emph{co...
Cooperation emergence is a central problem in multi-agent systems because decentralized agents must coordinate while adapting to the changing behavior of others. Evolutionary game theory identifies st...
Multi-constraint instruction following requires a model to respond to a query under many simultaneously active constraints. Even strong instruction-tuned models still routinely violate some of them. E...
arXiv:2609.26780v1 Announce Type: cross Abstract: Long-term conversational memory in multi-party settings requires more than retrieving relevant content from long-term conversations: it must distingu...
ForeDrive introduces a planning-relevant latent world model that is asymmetrically coupled to a Diffusion Transformer planner. The model learns multi‑horizon latent futures with a JEPA‑style world model, while planning gradients update the shared encoder and stop‑gradient routing trains the predictor with forecasting losses only. Gated visual fusion, future‑status injection, and Trajectory‑Adaptive Bias are used to guide trajectory generation without overriding current observations, achieving high performance on NAVSIM benchmarks using only front‑view images and pure imitation learning.
ToolCUA is an end‑to‑end agent that learns to optimally orchestrate GUI actions and tool calls for Computer Use Agents. It introduces a staged training pipeline that first scales interleaved GUI‑tool trajectories from static GUI data, then bootstraps decision making with single‑turn reinforcement learning, and finally refines performance via online agentic RL guided by a tool‑efficient path reward. On the OSWorld‑MCP benchmark, ToolCUA achieves 46.85% accuracy, a 66% relative improvement over the baseline and a 3.9% gain over GUI‑only models, setting a new state of the art for comparable‑scale models.
The paper introduces a novel end‑to‑end, size‑agnostic graph reinforcement learning framework for the one‑dimensional bin packing problem (1D‑BPP). It models packing as a Markov decision process on an item‑compatibility graph, where a graph neural network actor‑critic policy learns to merge compatible partial bins. Empirical results on the BPPLIB benchmark show that the learned policy reduces the mean optimality gap of a constructive heuristic from 2.66 % to 2.31 %, performs competitively against other learned methods, and outperforms a state‑of‑the‑art learned solver on the hardest benchmark family.
The paper introduces a black-box defense strategy for smart meter data that uses a proxy-guided hierarchical reinforcement learning framework to generate battery-based load-shaping policies. These policies inject realistic yet misleading appliance-level signatures into aggregate power signals, disrupting non-intrusive load monitoring attacks. Experiments on UK-DALE and REDD datasets show significant increases in appliance-level reconstruction error and reductions in attacker F1 scores across multiple unseen NILM models.