Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2610.28361v1 Announce Type: new Abstract: Long-horizon deep search requires LLM agents to iteratively decompose complex information needs, gather evidence from external sources, and maintain a...
arXiv:2610.26914v1 Announce Type: cross Abstract: Pluralistic alignment requires AI systems to learn from diverse populations whose preferences may reflect heterogeneous or even conflicting objective...
arXiv:2610.26932v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in non-verifiable domains such as financial services, where they must int...
arXiv:2610.27026v1 Announce Type: cross Abstract: Traffic cameras support useful analysis, but their outputs can reveal private information. Replacing images with text reduces direct visual exposure....
arXiv:2610.27516v1 Announce Type: cross Abstract: Despite advances in photorealism, semantic fidelity, and aesthetics, text-to-image generation and image editing remain understudied in a human-center...
arXiv:2610.28197v1 Announce Type: cross Abstract: Object-centric reinforcement learning agents typically receive each object's position in a fixed global frame, even though their decisions usually de...
arXiv:2610.28386v1 Announce Type: cross Abstract: Visual tool use enables vision-language models (VLMs) to acquire task-relevant evidence, but its effectiveness can be limited by predefined visual op...
arXiv:2610.28556v1 Announce Type: cross Abstract: Large language models (LLMs) can leverage semantic feature names alongside summary statistics for tabular feature selection, yet it remains unclear w...
arXiv:2610.28586v1 Announce Type: cross Abstract: Agentic on-policy distillation (OPD) trains a student agent on its own trajectories while a teacher scores every token. A natural explanation of wher...
arXiv:2410.03246v3 Announce Type: replace-cross Abstract: Deep Reinforcement Learning (DRL) enables robots to learn complex behaviors through interaction with the environment. However, due to the unr...
The paper surveys multimodal generative models (MGMs), outlining six main text‑conditioned modalities and four key techniques—self‑supervised learning, mixture of experts, reinforcement learning from human feedback, and chain‑of‑thought prompting—used across them. It evaluates models on faithfulness, compositionality, and robustness, and reports four empirical findings about preference gains, reward optimisation, intermediate planning, and expert granularity. The study also discusses trustworthiness risks such as bias, privacy leakage, and media misuse, and reviews mitigation evidence.
arXiv:2609.35767v2 Announce Type: replace-cross Abstract: Unified multimodal models can both look at and render images, so in principle they can repair their own generations: diagnose what an image g...
arXiv:2610.25916v2 Announce Type: replace-cross Abstract: As coding models and agent harnesses become more capable, reliable security verification is becoming a central bottleneck. Improving secure c...
LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks. We propose Self-Credit, an end-to-end a...
Asked to name a running shoe brand, language models say Nike. Asked to recommend one, they say Brooks. This paper asks where models learn that taste. Across 102 models and 42 brand categories, the bra...
The paper proposes a centralized training with decentralized execution framework for multi‑agent UAV pathfinding in stochastic environments, leveraging multi‑agent reinforcement learning enhanced by model predictive control concepts. Agents communicate only with a centralized planner and use a distance‑weighted mean‑field observation to plan feasible paths, while multi‑step value convergence improves training efficiency. Experiments, including ablation and real‑robot studies, demonstrate the method’s effectiveness and generalization.
The paper investigates emergent misalignment in reward models, showing that fine‑tuning reward models to prefer harmful content in a single domain (medical or financial) causes the preference to generalize to seven other domains. The authors fine‑tune two state‑of‑the‑art reward models—Skywork‑Reward‑V2‑Llama‑3.1‑8B and Skywork‑Reward‑V2‑Qwen3‑8B—and find that the models still perform well on the standard RewardBench 2 evaluation while exhibiting a preference for harm. Mechanistically, the study identifies two key changes in the reward models’ representations: a geometric compression that reduces the angle between sound and harmful responses, and a content‑specific rotation that aligns harmful representations more closely with the scalar head, thereby increasing their reward score.
The paper introduces RecReg, a safe reinforcement learning framework that enforces non‑Markovian regular safety properties through recovery‑based shielding. It provides theoretical foundations and rigorous safety and convergence guarantees in the tabular setting, and demonstrates empirical effectiveness using the MASA‑Safe‑RL library. The authors also outline an extension of RecReg to deep RL with state‑of‑the‑art model‑based methods.
The paper introduces Self-Contrastive Guidance (SCG), a method that contrasts a model’s predictions on real versus self-generated samples to directly guide generation toward the true data manifold. SCG is incorporated into the likelihood objective through reparameterization, resulting in the Self-Contrastive Alignment for Likelihood Estimation (SCALE) framework, which produces SCG-enhanced predictions in a single forward pass. Experiments on class-conditional and text-to-image generation with diffusion and autoregressive models show that SCALE consistently outperforms existing guidance-free baselines and further benefits from combining with Direct Preference Optimization (DPO).
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning proposes TreeOPSD, a critic‑free framework that treats each interaction turn as a tree node, branches alternative continuations, and recursively backs up terminal rewards to generate implicit turn‑level credit. The method maps this credit to a bounded teacher distribution based on the likelihood ratio to a frozen behavior policy, and minimizes the divergence between teacher and student to recover policy‑gradient directions while limiting updates across mini‑batches. Empirical results show TreeOPSD enhances agentic performance on long‑horizon benchmarks, notably improving Qwen3‑30B‑A3B‑Thinking‑2507’s accuracy by 4.97 points over GRPO in the Deep Research setting.