Hugging Face Trending Papers
QF3: Fast Flow RL with Filtered Q-Gradients
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
6d ago
QF3: Fast Flow RL with Filtered Q-Gradients
QF3 (Fast Flow RL with Filtered Q-Gradients) is an online off‑policy reinforcement learning algorithm that trains flow policies by combining flow matching with the critic’s action gradient, backpropagated through a one‑step prediction of the flow output. It selectively applies the critic gradient only to action dimensions close to the replay action to ensure reliable updates. QF3 is the first off‑policy flow RL method to train humanoid locomotion policies from scratch and transfer them zero‑shot to hardware, achieving a 10× wall‑clock speedup over the on‑policy FPO++ method, and it can also fine‑tune pretrained flow‑based manipulation policies on ABC‑Sim and Robomimic tasks.
arXiv Machine Learning
5d ago
Q-Learning with Scalar Adjoint Matching
The paper introduces Q‑Learning with Scalar Adjoint Matching (SQAM), a method that replaces expensive per‑step vector–Jacobian products in flow policy fine‑tuning with a closed‑form scalar adjoint that scales the value gradient by flow time. SQAM also incorporates a value penalty on policy‑generated actions, which is shown to be crucial for effective learning. Experiments demonstrate that SQAM outperforms strong baselines on the hardest OGBench domains and improves supervised fine‑tuning on a real bimanual robot with a vision‑language‑action policy.
arXiv AI
Jun 10
Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning
arXiv:2606. 11087v1 Announce Type: cross Abstract: Expressive continuous control policies, such as diffusion and flow models, form the backbone of recent advances in scaling imitation learning for simulated and real robot control.
arXiv Machine Learning
Jul 31
REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning
arXiv:2603. 13707v3 Announce Type: replace-cross Abstract: Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks.
arXiv AI
Aug 18
Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning
arXiv:2608. 15088v1 Announce Type: cross Abstract: Human-in-the-loop (HIL) online reinforcement learning for real robots must absorb human interventions quickly while continuing to improve beyond the human prior.
arXiv AI
Jun 17
When Life Gives You BC, Make Q-functions: Extracting Q-values from Behavior Cloning for On-Robot Reinforcement Learning
arXiv:2605. 05172v2 Announce Type: replace-cross Abstract: Behavior Cloning (BC) has emerged as a highly effective paradigm for robot learning.