← Back to all news
arXiv Machine Learning By Baha Zarrouki, Arslan Thobani, Jasper Hoffmann, Mattia Piccinini, Rudolf Reiter, Felix Jahncke, S\'ebastien Gros, Davide Scaramuzza, Johannes Betz

Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPC

Read the original on arXiv Machine Learning →

The paper introduces Solver-Gradient Guided Reinforcement Learning (SG‑RL), a method that augments standard RL with bounded gradients from a differentiable MPC solver to adapt cost‑function weights online. SG‑RL integrates solver‑gradient guidance into PPO through actor‑update scaling, policy loss, advantage estimation, and value‑function learning, achieving comparable or superior closed‑loop performance while requiring up to 70.6% fewer samples. Experiments on two autonomous racing platforms with intentional model mismatch demonstrate that SG‑RL outperforms both RL and gradient‑based policy learning baselines and generalizes zero‑shot to unseen environments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 5

Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learning

The paper introduces Multi‑Fidelity Policy Gradient (MFPG) methods that combine scarce, high‑fidelity (HF) reinforcement‑learning data with abundant, low‑fidelity (LF) data to stabilize training. Unlike prior approaches that directly optimize biased LF objectives, MFPG uses LF data only to construct a control variate that reduces variance without biasing the policy gradient. The authors extend MFPG to modern actor‑critic algorithms, notably a budget‑aware MFPG‑PPO that preserves cross‑fidelity correlation, monitors estimator uncertainty, and allocates a fixed sampling budget between HF and LF sources. Experiments on simulated robot locomotion and a physical Franka arm show that MFPG‑PPO consistently outperforms standard PPO trained on HF data alone and matches performance achieved with 16× more HF data on the hardest tasks, while enabling stable learning with as few as four real‑robot episodes per update.

By Xinjie Liu, Ruihan Zhao, Anirban Chaudhuri, Cyrus Neary, Ufuk Topcu, David Fridovich-Keil
reinforcement-learningrobotics
More like this →