arXiv Machine Learning

MA-UITD: Offline Multi-Agent Reward Inference for Characterizing Continuous Social Behavior

arXiv AI
6d ago

Learning Intrinsic Dynamics in Latent World Models

The paper investigates how encouraging action predictability in latent world models can preserve intrinsic dynamics, which are crucial for planning. By deriving a theoretical lower bound and conducting empirical tests, the authors show that action predictability prevents the encoder from merging distinct action outcomes, even when visual appearance varies. Using a method called Concat IDM, they achieve a substantial increase in planning success on the Push‑T benchmark, from 9.1% to 77.2%, demonstrating the practical benefit of this inductive bias.

By Jiamin Shi, Yuanhong Zhang, Zhiyang Liu, Jiawei Sun, Jingwen Fu
arXiv Machine Learning
Oct 5

Learning Transferable Policies from Action-free Time Series Through Dynamical Embeddings

The paper introduces a hierarchical model-based reinforcement learning framework that learns control policies from action-free time series by exploiting shared dynamics across related systems. It uses low-dimensional dynamical embeddings to capture both shared structure and individual variation, which then parameterize shared policy and value networks. Experiments on Lorenz-63, double-pendulum, and neural-behavioral data show that these hierarchical policies outperform independently trained ones, achieve higher rewards than planning with the same models, and generalize to unseen systems using only embedding inference.

By Niklas Emonds, Georgia Koppe
arXiv Machine Learning
Jul 8

Supervised Reward Inference

arXiv:2502. 18447v2 Announce Type: replace Abstract: Existing approaches to reward inference typically assume that humans provide demonstrations according to specific behavior models.

By Will Schwarzer, Jordan Schneider, Philip S. Thomas, Scott Niekum
arXiv AI
Oct 9

Q-Shaped Options for Hierarchical Reinforcement Learning

The paper introduces Q-Shaped Options (QSO), a hierarchical reinforcement learning framework that learns action abstractions between consecutive levels using shared encoders shaped by Q functions. QSO employs distinct state-value functions, Q functions, and policies at each hierarchy level, encouraging the low-level Q function to preserve distinctions needed for optimal control while the high-level Q function discards unnecessary distinctions. Experiments on offline goal-conditioned locomotion and manipulation tasks show that QSO learns semantically meaningful option spaces and outperforms baselines, achieving non-zero performance where other algorithms fail.

By Clarisse Wibault, Antoine Gorceix, Antonio L\'eon Villares, Alexey Zakharov, Evangelos Chatzaroulas, Michael Matthews, Eduardo Pignatelli, Jakob Foerster