← Back to all news
arXiv AI By Panagiotis Roditis, Panagiotis P. Filntisis, Petros Maragos

A Geometric Approach to Soft Actor-Critic with Zonotopes for Locomotion Learning

Read the original on arXiv AI →

The paper introduces GeZo‑SAC, a variant of Soft Actor‑Critic that incorporates geometric representations (zonotopes) to adapt critic pessimism based on state and action. Each critic predicts a zonotope whose width is subtracted from the critic value as a pessimistic offset, and the disagreement between critics is measured and used to blend the critics via a log‑sum‑exp aggregation. Experiments on MuJoCo‑v5 locomotion tasks show GeZo‑SAC achieving the highest mean return on Ant‑v5 and Hopper‑v5, while also reducing actuator work and action effort and maintaining near‑zero overestimation frequency.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Aug 28

Shared Actors Need Not Share Critics: Effects of Value Mismatch in Parallel Reinforcement Learning

The paper investigates the problem of sharing a single critic across multiple parallel environments in reinforcement learning. It shows that when environments assign different expected returns to the same state, a shared critic must reconcile conflicting value targets, which can distort advantage estimates and misguide policy updates. The authors propose a simple fix—providing the critic with the environment index—demonstrating through bandit models and experiments on CartPole, MuJoCo, BipedalWalker, and 16 Procgen games that this conditional critic stabilizes learning and boosts returns, achieving a 40.8% improvement in aggregate normalized return on unseen levels.

By Zhenya Liu, Yang Meng, Zhuokai Zhao, Xuefeng Liu, Yuxin Chen
reinforcement-learning
More like this →
arXiv Machine Learning
Sep 14

Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning

The paper introduces GACA, a critic‑free reinforcement learning estimator that adapts credit assignment granularity based on a step‑level uncertainty proxy. GACA assigns higher weight to fine‑grained signals for steps with above‑average negative log‑likelihood, while relying on episode‑level signals for less uncertain steps, improving task success on ALFWorld and WebShop for 1.5B and 7B language models. The authors provide a risk decomposition, a conditional bound on action‑value variation, and an error‑projection analysis to justify the method’s effectiveness.

By Taoran Liang, Yang Liu, Shang Luo, Yingguang Yang, Rongrong Zhang, Yingzong Min, Yulin Huang, Jianshen Zhang, Yongzhi Qi, Kefu Xu, Congjing Ran, Bin Chong
llmsagentsreinforcement-learningsafety
More like this →