← Back to all news
arXiv AI By Oriol Corcoll, Qiyao Wei, Ciar\'{a}n Mark Gilligan-Lee

Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning

Read the original on arXiv AI →

The paper introduces Explanatory World Models (EWM), an attribution framework that leverages explanations from a frontier language model to identify earlier events that plausibly contributed to a delayed outcome. EWM redistributes reward recursively while preserving the undiscounted episode return, and is evaluated on long‑horizon RL benchmarks, a public sequential‑recommendation task, and an industry‑scale generative recommender, consistently outperforming baseline methods, especially when rewards are highly delayed. The study demonstrates that structured explanations can provide reusable credit signals for learning from delayed outcomes.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
4d ago

Beyond Sequences: Distilling Structured Decision Memory for LLM Recommendation

The paper introduces MARI, a Memory-Augmented Recommendation system that uses a Decision Memory Bank to store Structured Decision Memories (SDMs) capturing users’ goals, constraints, and trade-offs. Unlike prior methods that flatten heterogeneous behaviors into token sequences, MARI retrieves relevant SDMs to enhance LLM reasoning, improving performance on next-item prediction and a new Difficult Choice Prediction task while keeping latency low. The approach also provides interpretable, human-readable insights into user decision-making.

By Leikun Liang, Guoshuai Wang, Xingsheng He, Yushan Han, Yunyi Xuan, Xiaoxiao Xu, Lin Qu
llmsefficiencybenchmarkssafety
More like this →
arXiv AI
Sep 15

Safety as a Constraint: Fine-Tuning a LLM Recommender to Explain Itself

The paper presents a method for fine‑tuning a large language model (LLM) recommender to generate personalized, non‑harmful explanations for its recommendations. By training two LLM‑judge reward models and using constrained GRPO, the authors achieve a significant increase in the PASS rate for all three criteria, from 0.649 to 0.956 on their own judges and from 0.677 to 0.931 on an independent judge. The fine‑tuned model maintains its original recommendation performance, demonstrating that LLM‑based recommenders can be adapted to complex tasks without loss of effectiveness.

By Jiashu He, Emma Yanyang Kong, JJ Tan, David Fagnan
llmsagentsfine-tuningsafety
More like this →
arXiv Machine Learning
4d ago

CausalDreamer: Learning Predictive World Models with Latent Disentanglement

CausalDreamer enhances generative world models by freezing the original video tokenizer and re-encoding its latent space into a factored representation that separates controllable versus uncontrollable dynamics and reward-relevant versus reward-irrelevant information. The pretrained dynamics model is fine‑tuned to predict this factored representation, enabling more accurate model‑predictive planning. On the MMBench2 benchmark, CausalDreamer improves normalized scores by 14% on clean tasks and 25% on manipulated variants compared to the baseline, while both models perform poorly on entirely new environments.

By Prince Jha, Nils Lukas, Kun Zhang, Salem Lahlou
agentsnlpfine-tuning
More like this →
arXiv AI
Aug 18

Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation

arXiv:2608. 11980v2 Announce Type: replace-cross Abstract: Semantic-ID generative recommenders represent each item as a short sequence of discrete semantic tokens and predict the next item by autoregressively generating this token sequence.

By Kangning Zhang, Haotian Fang, Xukun Luo, Hao Yin, Yang Gao, Peng Yan, Weiwen Liu, Weinan Zhang, Yong Yu
reinforcement-learningfine-tuningbenchmarkssafety
More like this →