arXiv Machine Learning

CARE-VI: Conservative Adaptive Reliability Estimation for Value Improvement in Off-Policy Actor-Critic Learning

CARE‑VI introduces a framework for improving value targets in off‑policy actor‑critic learning by combining Conservative Adaptive Ranking and Screening (CARS), Selector‑Evaluator Value Assessment (SEVA), and Dynamic Adaptive Risk‑aware Enhancement (DARE). CARS limits candidate actions to a budgeted prefix and expands it only when uncertainty exceeds a threshold; SEVA orders candidates with selector critics and reviews their values with an evaluator critic, capping the value at the selector reference; DARE adjusts residual corrections based on candidate reliability and signal gaps. Theoretical analysis bounds errors in each component, and empirical tests on SAC, TD3, and TD7 across four MuJoCo tasks show CARE‑VI consistently outperforms baselines in mean return.

arXiv AI

Sep 1

BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning

BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.

By Dongsheng Hou, Yanqiao Chen, Yuhan Rui

reinforcement-learning safety

arXiv Machine Learning

Jul 30

Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning

arXiv:2607. 26509v1 Announce Type: new Abstract: Deep off-policy reinforcement learning algorithms for continuous control typically rely on neural value function approximation to guide policy improvement.

By Gong Gao, Xiao Lai, Ziqi Xie, Guojie Chen, Xianhui Liu, Weidong Zhao

reinforcement-learning safety

arXiv Machine Learning

Aug 11

CODS: Iterative Bellman-Residual Data Selection for Reusable Offline Reinforcement Learning

arXiv:2608. 07719v1 Announce Type: new Abstract: Offline reinforcement learning repeatedly trains policies from a fixed transition pool, making redundant data costly across seeds and hyperparameters, while naive subsampling can remove rare transitions needed for long-horizon credit assignment.

By Ibne Farabi Shihab, Sanjeda Akter, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus Swaqeeb, Anuj Sharma

reinforcement-learning

arXiv Machine Learning

Aug 26

Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning

arXiv:2608.24858v1 Announce Type: new Abstract: Marginalized importance weighting evaluates a target policy by reweighting offline state-action samples with its discounted occupancy ratio, characteri...

By Lars van der Laan, Nathan Kallus

reinforcement-learning

arXiv Machine Learning

Aug 12

IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning

arXiv:2608. 10634v1 Announce Type: new Abstract: Model-based reinforcement learning (MBRL), which learns environment dynamics to generate synthetic experience, is a promising approach to sample-efficient decision making.

By Zefeng Liang, Jie Qiao, Ruichu Cai, Weilin Chen, Zhifeng Hao

reinforcement-learning safety

arXiv Machine Learning

Aug 4

CARE: Context-Aware Ranking Evolution with Executable Scoring Programs for Budgeted Reaction Optimization

arXiv:2606. 14581v4 Announce Type: replace Abstract: High-throughput experimentation can evaluate many reaction conditions, yet combinatorial condition spaces still exceed the available experiment budget.

By Guanyu Liu, Weiyi Kong, Chao Tang, Zeyu Wang, Boer Zhang, Baiqing Li, Peiyu Zhang, Tianyu Shi

llms efficiency

Hugging Face Trending Papers

Aug 25

Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning

Marginalized importance weighting evaluates a target policy by reweighting offline state-action samples with its discounted occupancy ratio, characterized by an adjoint Bellman equation. Existing mini...

reinforcement-learning

arXiv Machine Learning

Aug 24

Decoupling Policy Extraction for Offline Reinforcement Learning

arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...

By Xuyao Lin, Yixiang Shan, Jinru Duan, Tao Yang, Xinyu Zhao, Runyu Lei, Yiming Zhao, Jiaxin Fan, Zongbao Feng, Peng Jia

rag reinforcement-learning

arXiv Machine Learning

Jul 31

Good Rankers, Bad Objectives: Bilinear Contrastive Critics under Expressive Policy Search

arXiv:2607. 27422v1 Announce Type: new Abstract: Good action rankings do not make a contrastive critic safe to maximize.

By Ayushman Singh, Siddharth Aphale

rag robotics

arXiv Machine Learning

Jul 10

Beyond Success Rates: Trainability and Extractability for Offline GCRL

arXiv:2602. 05459v2 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method.

By Jan Malte T\"opperwien, Aditya Mohan, Marius Lindauer

reinforcement-learning robotics

arXiv AI

Jun 9

Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning

arXiv:2606. 08735v1 Announce Type: new Abstract: Quality-diversity reinforcement learning (QD-RL) aims to construct policy repertoires that contain both high-performing and behaviorally diverse policies.

By Lianrong Zuo, Peilan Xu, Yong Liu, Wenjian Luo

reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 16

Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes

arXiv:2606. 17043v1 Announce Type: cross Abstract: When pretrained VLA policies are fine-tuned through online RL, each rollout episode produces only a single binary outcome (success or failure), yet the actor update requires per-transition supervision.

By Tongyan Fang, Siyuan Huang, Naiyu Fang, Ganlong Zhao, Zhongjin Luo, Jianbo Liu, Xiaogang Wang, Ying Dong, Hongsheng Li

agents reinforcement-learning robotics fine-tuning

Related stories