Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2609.39578v1 Announce Type: new Abstract: Agent harnesses often improve language models with human-designed workflows, but as models grow more capable, unreliable guidance can increasingly cons...
arXiv:2609.39883v1 Announce Type: new Abstract: Video temporal grounding supports applications such as video search, content review, and automated editing by localizing events described in natural la...
Sparse triangular solve (SpTRSV) is a fundamental kernel in numerous scientific and engineering applications. However, the data dependencies inherent in sparse triangular matrices significantly limit...
Recent advances have enabled unified omni-modal models in understanding audio, vision, and language. However, existing benchmarks, training data, and learning methods largely treat the modalities inde...
This paper studies \textbf{thinking--answer consistency} in vision-language models. We focus on Visual Intention Grounding, where a model infers a target object based on a human intention query and pr...
We study infinite-horizon average-reward constrained Markov decision processes (CMDPs) under the weakly communicating assumption. Existing high-probability guarantees for this setting either require c...
Legal language models require reward signals that capture not only answer correctness but also the multidimensional quality of legal responses. Existing reward methods, however, often rely on coarse-g...
arXiv:2609.37105v1 Announce Type: cross Abstract: Language model agents can be improved by updating their model weights or refining the harness that guides task execution. These components are couple...
The paper introduces STRAT, an auxiliary task for deep reinforcement learning that trains agents to generate a short textual trace describing their state. This trace, produced online by environment rules, includes landmark, route, and survey information such as position, inventory, goals, and progress. Adding a single auxiliary head to a standard policy, STRAT improves performance on 60 sparse-reward XLand-MiniGrid tasks, compacting state representations, preventing rank collapse, and offering a readable account of agent beliefs at no extra cost.
CAR‑VLA is a Vision‑Language‑Action model for autonomous driving that jointly considers scene complexity and dynamic risk to determine reasoning depth, urgency, and focus. It maps four complexity‑risk categories to three reasoning modes—Fast Intuition, Slow Thinking, and Reflex Response—each tailored to different driving scenarios. The model is trained via progressive supervised learning and reinforcement learning, achieving competitive performance on NAVSIM and Navhard benchmarks and demonstrating risk‑aware reasoning in high‑risk scenarios.
Beacon is a new agentic visual reasoning model that improves multimodal large language models (MLLMs) by better deciding when to use tools and how to use them. It introduces two key concepts—Mode Adaptiveness, which ensures tools are invoked only when necessary, and Tool Effect, which measures the net benefit of tool use— and trains the model with supervised fine‑tuning and reinforcement learning that rewards necessity-aware decisions and expands capability through expert hints. Across 13 benchmarks, Beacon outperforms other open‑source models, achieving the highest average score and the largest net tool‑gain on diagnostic tests.
The paper investigates whether input ablations on predictive models can reliably reveal the importance of information for explaining human sequential choice behavior. Using two synthetic bandit tasks with known generating policies, the authors compare GRUs, Transformers, a fine‑tuned LLaMA, and cognitive models under varied reward contributions. They find that while neural models can predict choices well, their responses to ablations often diverge from the true generating process, indicating that predictive accuracy alone does not guarantee faithful model ablations.
RelayVSR introduces a streaming video super‑resolution framework that combines a large generative model, which produces reference latents for sparse keyframes, with a lightweight Dual‑Memory Video Transformer that super‑resolves every frame using these references and low‑resolution input. The method employs Video‑Aware Reference Optimization (VARO), a reinforcement‑learning strategy that optimizes both system‑level video quality and reference‑level keyframe fidelity, outperforming direct joint training. On 1080p video, RelayVSR achieves 29.29 FPS with modest GPU memory usage, significantly faster and more efficient than the FlashVSR‑Tiny baseline.
VISTA is a method for active multimodal agents that internalizes collective visual experience via on‑policy distillation. It turns observations from multiple rollouts of the same input into shared supervision, using Collective Visual Experience Distillation (CVED) to organize observations with context and Heterogeneity‑Aware Policy Improvement (HAPI) to reinforce successful trajectories and guide learning from unsuccessful ones. The approach lets an experience‑conditioned teacher evaluate a student’s partial responses, enabling discoveries from one trajectory to inform others without altering the student’s original history, and achieves superior performance on fine‑grained perception and general reasoning tasks compared to comparable agents.
arXiv:2605.30327v2 Announce Type: replace-cross Abstract: Frontier reasoning models are produced by post-training base language models with reinforcement learning. Recent work has challenged this by...
The paper introduces Dually Regularized AIL, a model‑free algorithm for adversarial imitation learning that jointly applies KL policy regularization and a quadratic reward penalty based on expert and learner occupancies. It proves fast convergence rates, achieving a ×O(1/K+1/N) bound on the regularized imitation gap in finite‑horizon MDPs with general function approximation, and establishes the first algorithm to attain ×O(1/ε) sample complexity in both expert demonstrations and online interactions for this regularized objective.
arXiv:2609.35793v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly trained with reinforcement learning from verifiable rewards (RLVR). An exact verifier can also support te...
arXiv:2609.37200v1 Announce Type: new Abstract: Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary...
arXiv:2609.35750v2 Announce Type: replace-cross Abstract: Scaling the horizon of agentic LLMs is bottlenecked by the need to fit ever longer context traces in GPU memory. Context compaction has been...
arXiv:2609.36934v1 Announce Type: new Abstract: Traffic signal control (TSC) is essential for improving urban mobility and reducing congestion. Although roadside cameras are widely deployed at signal...