← Back to all news
arXiv Machine Learning By John L. Zhou, Yuxuan Dong, Jonathan C. Kao

An Informational Curse of Horizon in Goal-Conditioned Policy Learning

Read the original on arXiv Machine Learning →

The paper identifies an informational curse of horizon in goal-conditioned policy learning, showing that longer goal relabeling horizons during training can hurt policy generalization and performance. Controlled experiments with oracle planners separate training horizons from test horizons, revealing that behavioral cloning policies degrade with longer horizons while reinforcement learning mitigates this effect. The authors attribute this to reduced conditional mutual information between actions and hindsight-relabeled goals, and demonstrate that distilling short-horizon policy Jacobians into long-horizon policies improves performance, especially in combinatorial manipulation tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 25

Policy Complexity, Reaction Time, and Bounded Rationality in Reinforcement Learning

The paper introduces MI‑SARSA, an on‑policy temporal‑difference algorithm that incorporates mutual‑information regularization to model bounded rationality in reinforcement learning. By penalizing state‑specific deviations from a learned marginal action prior, the algorithm selectively uses state information only when the expected return outweighs the informational cost, yielding a reward‑complexity tradeoff. MI‑SARSA also predicts reaction times, showing that stronger information penalties lead to simpler policies, lower control costs, and faster responses, while regularization mitigates performance loss after environmental shifts at the expense of asymptotic return.

By James Wu, Chris R. Sims
agentsreinforcement-learning
More like this →
arXiv Machine Learning
Sep 24

Limiting-Kernel Q($\lambda$): Bridging Short and Long Horizons

Limiting‑Kernel Q(λ) (LKQL) is an off‑policy value estimator that blends n‑step truncation with a long‑horizon approximation based on the limiting kernel. It maintains the computational efficiency of n‑step methods while improving policy evaluation accuracy, especially for long‑horizon tasks. The authors prove faster convergence of LKQL’s operator under aperiodicity and near‑on‑policy conditions, and demonstrate empirical gains on MuJoCo continuous‑control benchmarks.

By Tolga Ok, Arman Sharifi Kolarijani, Peyman Mohajerin Esfahani, Mohamad Amin Sharifi Kolarijani
reinforcement-learningbenchmarks
More like this →