arXiv AI By Riccardo Barbano, Vincent Pauline, Runchang Li, George Webber, Alexander Denker, \v{Z}eljko Kereta, Stefan Bauer, Francisco Vargas, Esmeralda S. Whitammer

One for All, All for One: Coordinated Multi-Agent Diffusion Steering via Stochastic Optimal Control

Read the original on arXiv AI →

The paper introduces Coordinated Multi-Agent Diffusion Steering (CMDS), a framework that reuses independently trained diffusion models as generative primitives and learns a control policy to coordinate their reverse processes. By framing coordination as a stochastic optimal control problem, CMDS balances an assembly-level reward with adherence to pretrained dynamics, enabling the learned control to generalise across new tasks. Experiments demonstrate CMDS’s ability to recover target distributions, satisfy spatial constraints, and separate sources from degraded mixtures in applications such as maze navigation, robot planning, and text-conditioned human motion.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 16

Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond

The paper reviews how reinforcement learning (RL) can refine diffusion models (DMs) after pretraining, addressing limitations in reasoning, exploration, and alignment. It surveys RL-driven DMs across language, multimodal, robotic, and scientific domains, focusing on trajectory optimization and reward engineering. Experiments highlight the benefits of various RL post‑training strategies and identify key bottlenecks—reward fragility, exploration‑exploitation trade‑offs, and training instability—while proposing a roadmap for future research and a public repository for updates.

By Zhiyuan Ma, Jiabao Wei, Tianyu Wang, Hanqing Wang, Jiadong Pan, Haowen Xu, Liangliang Zhao, Ruixun Liu, Yuhan Wang, Yuwei Zhang, Liang Feng, Junhao Xiao, Kaiyan Zhang, Xuekai Zhu, Daizong Liu, Biqing Qi, Yihao Liu, Ganqu Cui, Ning Ding, Linfeng Zhang, Jianjun Li, Kun He, Xiang Bai, Bowen Zhou