arXiv Computer Vision By JunGyu Lee, Jisu Shin, Seunghyun Shin, Hae-Gon Jeon

Controllable Crowd Generation through World-Model Planning

Read the original on arXiv Computer Vision →

The paper introduces Ctrl-CWM, a multi‑agent controllable crowd world model that merges crowd generation with run‑time control. It uses an encoder to learn human motion dynamics, an actor to propose pedestrian displacements, a critic to evaluate imagined trajectories, and a planner that selects actions based on critic scores and user‑defined costs. Experiments show that Ctrl‑CWM outperforms state‑of‑the‑art methods in realism and collision metrics while allowing new control objectives to be added without retraining.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Machine Learning
Aug 28

Diffusion Policies for Short-Horizon Planning in Robot Crowd Navigation

The paper introduces Planning Diffusion Policy Optimization (PDPO), an offline‑to‑online reinforcement‑learning framework that employs a diffusion policy to produce short‑horizon action chunks for robot crowd navigation. PDPO is pretrained on collision‑avoidance demonstrations and fine‑tuned online with PPO, generating five‑step action sequences applied in a receding‑horizon manner. The authors also identify a benchmark artifact where agents can leave the valid domain without explicit boundary constraints, and they mitigate this by treating boundary violations as collisions, leading to improved success rates over strong baselines.

By Wendong Li, Jochen Garcke
arXiv Computer Vision
Aug 25

BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation

arXiv:2608.22187v1 Announce Type: cross Abstract: Modern driving action models are increasingly improved in a self-improvement loop, where a learned world simulator imagines future observations and t...

By Jiaqi Wang, Zhuo Zhang, Haining Guan, Tingguang Zhou, Haowen Cui, Zhongyang Zhu, Yulong Zheng, ChuanYe Wang, Xuefeng Chen, Zhen Yang, Tianchen Deng, Feiyang Tan, Hangning Zhou, Bo Dai, Lixia Shen, Xiwu Chen, Xiyang Wang, Jiajun Zhu
arXiv Computer Vision
3d ago

SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models

SimVTA is a post‑training framework that enhances driving world action models by aligning simulated future videos with candidate trajectories. It samples trajectories, renders high‑reward futures in a real‑world simulator using 3D Gaussian Splatting, and prioritizes video supervision for high‑error futures. The approach improves planning performance, raising PDMS from 86.2 to 91.3 on NAVSIM v1 and achieving strong results on NAVSIM v2 and HUGSIM.

By Changhui Jing, Haochen Tian, Hongchen Li, Tianyu Li, Caojun Wang, Zihao Zhang, Yanhao Liu, Chengen Xie, Bailin Li, Hao Zhao, Zhenjie Yang
arXiv AI
Aug 24

WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving

arXiv:2608.20974v1 Announce Type: cross Abstract: Video Joint Embedding Predictive Architecture (V-JEPA) learns powerful spatiotemporal representations from video through self-supervised latent featu...

By Xinlin Wang, Yujiao Xiang, Yuheng Zhou, Jingqi Wang, Minqing Huang, Jiajie Huang, Dongxu Wei, Tingguang Zhou, Xiyang Wang, Gong Chen, Zhi Xu, Feiyang Tan, Hangning Zhou, Mu Yang