arXiv Machine Learning
Aug 28

Arrive and Survive: Scaling Safe Goal-Conditioned Policy Learning from One-Bit Failure Signals

The paper introduces Safe Contrastive Reinforcement Learning (Safe-CRL), a method that corrects bias in contrastive RL caused by failure-terminated Markov decision processes. By applying mass-weighted InfoNCE and a log-survival-mass score, Safe-CRL uses only a one-bit failure signal to improve survival and goal-reaching performance across twelve robot navigation and locomotion tasks. The approach demonstrates complex failure-avoidance behaviors and completes the theoretical foundation of contrastive RL under failure termination.

By Guopeng Li, Yiyang Duan, Yiru Jiao, Chengcheng Xu
arXiv AI
Oct 28

Data-Driven Maintenance Optimization: Aligning Prognostic Learning with Decision Objective

The paper investigates data‑driven predictive maintenance where health‑condition estimation is coupled with an ordering‑replacement decision model. It shows that the traditional Estimate‑Then‑Optimize (ETO) approach can lead to inconsistent, suboptimal decisions and introduces an Integrated Estimate‑Optimize (IEO) framework that directly optimizes maintenance cost, bypassing conventional accuracy metrics. Experiments on CMAPSS data demonstrate that IEO reduces mean maintenance cost by 14.8–22.7% versus ETO and 7.2–11.8% versus a decision‑oriented baseline, while also lowering failure rates and cost variability, even when prediction errors are higher.

By Zhuojun Xie, Adam Abdin, Yiping Fang
arXiv AI
Sep 3

PGPO: Potential-Guided Policy Optimization for Multi-Turn Agentic Tasks

The paper introduces Potential-Guided Policy Optimization (PGPO), a method for multi-turn agentic tasks that improves credit assignment by estimating empirical state potentials from anchor-state-group return statistics. PGPO derives action advantages from potential differences between adjacent states, enabling cross-trajectory credit propagation and finer-grained step-level credit assignment, especially within failed trajectories. Experiments on ALFWorld and WebShop demonstrate strong performance compared to recent group-based reinforcement learning methods, with negligible training overhead.

By Yuyao Zheng, Haipeng Sun, Junwei Bao, Lemao Liu, Hongfei Jiang, Yang Song, Dejing Dou
arXiv AI
Sep 30

Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents

The paper introduces Milestone Viability Potential Policy Optimization (MVPO), a reinforcement learning algorithm designed for long‑horizon large language model agents. MVPO addresses the zero‑credit failure problem by learning from viable failure prefixes, estimating prefix potential over Union‑Find viability regions, and repairing zero‑credit groups with potential‑difference advantages. Experiments on Qwen2.5‑1.5B‑Instruct demonstrate that MVPO outperforms eight strong baselines, improving success rates on ALFWorld and WebShop with minimal overhead.

By Qi Zhou, Yuanfan Li