← Back to all news
arXiv AI By Chenyuan Zhou, Yining Hong, Huang Huang, Yunfan Jiang, Wenlong Huang, David Held, Vitor Campagnolo Guizilini, Yue Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Yejin Choi

T2-z: Test-Time Optimization of Environment Latents for Robotic World Models

Read the original on arXiv AI →

T2-Z introduces a test‑time optimization framework that augments robotic world models with a learnable environment latent, enabling the model to adapt to unseen environmental conditions without explicit supervision of those factors. During training, the method alternates between optimizing the world‑model dynamics and the latent representation, allowing the latent to capture underlying environmental variation. At test time, a freshly initialized latent is optimized from a few interactions, reducing prediction error from 15.20 to 7.00 and increasing action success from 33.73% to 72.96% across simulated and real‑world tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 23

Feedback World Model Enables Precise Guidance of Diffusion Policy

The paper introduces a feedback world model that updates predictions online using the robot’s observed next state after each action, thereby closing the loop between prediction and observation. This lightweight feedback state corrects future predictions without extra training or parameter updates, and the authors provide convergence guarantees under bounded dynamics residuals. Experiments on LIBERO‑Plus, Robomimic, and real‑world tasks show significant reductions in prediction error (up to 76.4%) and improved policy performance, even when applied to pretrained foundation models like Fast‑WAM‑Joint.

By Tuo An, Jindou Jia, Gen Li, Jingliang Li, Chuhao Zhou, Pengfei Liu, Bofan Lyu, Jiaqi Bai, Xinying Guo, Geng Li, Jianfei Yang
diffusionrobotics
More like this →
arXiv AI
Aug 28

Predicting Consequences and Reinforcing Navigation Policies with Latent World Models

The paper introduces a Latent World Model (LWM) for robot navigation that predicts action‑conditioned latent feature compatibility instead of reconstructing future observations. By exploiting the correlation between spatial proximity and latent feature similarity, the model evaluates action consequences directly in latent space and supports counterfactual training using sampled action sequences. The learned world model can supervise policy learning from unlabeled video and further improve policies via reinforcement learning entirely within the model, eliminating the need for action annotations and additional environment interaction.

By Zengmao Wang, Wei Gao, Shuhan Shen
agentsdiffusionreinforcement-learningrobotics
More like this →