arXiv AI
When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RL
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 9
ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies
arXiv:2606. 09630v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies provide strong priors for language-conditioned manipulation, but remain brittle in off-nominal states requiring targeted recovery.
arXiv AI
Oct 15
LLM Value Functions for Credit Assignment in Agentic RL
arXiv:2610.15840v1 Announce Type: cross Abstract: Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assign...
arXiv AI
Jun 3
When RLHF Fails: A Mechanistic Taxonomy of Reward Hacking, Collapse, and Evaluator Gaming
arXiv:2606. 03238v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) makes large-scale post-training possible by replacing an underspecified human objective with learned and scalable proxies.
arXiv AI
Jun 4
Plan, Watch, Recover: A Benchmark and Architectures for Proactive Procedural Assistance
arXiv:2606. 04970v1 Announce Type: cross Abstract: We envision a proactive multi-modal assistant system which gives users real-time step-by-step guidance on a procedural task, autonomously deciding \textit{when} to interrupt, and \textit{how} to coach.
arXiv AI
Sep 4
FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience
arXiv:2609. 03241v1 Announce Type: cross Abstract: A reasoning model can improve from its own on-policy experience, but this inner loop is fragile: terminal verifiers provide reliable yet sparse supervision, while dense same-model guidance can reinforce false confidence or overconcentrate learning on a narrow solution mode.
arXiv Computation and Language
Sep 22
FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model
FLARE introduces a dense supervision paradigm for long‑horizon coding agents, leveraging a Generative Reward Model (GRM) trained via the RADAR diagnostic framework. The GRM provides real‑time, step‑level risk feedback, enabling FLARE to act as an active scaffold that intercepts high‑risk steps during inference and supplies structured signals for post‑training fine‑tuning and reinforcement learning. Experiments show FLARE outperforms existing methods, achieving a 5× reduction in token consumption and significant performance gains in both supervised fine‑tuning and RL settings.