arXiv Computer Vision
Oct 21

HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

HERMES++ is a unified driving world model that simultaneously performs 3D scene understanding and future geometry prediction. It uses a BEV representation to merge multi‑view spatial data for LLM compatibility, introduces LLM‑enhanced world queries for knowledge transfer, and employs a Current‑to‑Future Link to condition future geometry on semantic context. A Joint Geometric Optimization strategy enforces structural integrity by combining explicit geometric constraints with implicit latent regularization, and extensive benchmark tests show that HERMES++ outperforms specialist methods in both point cloud prediction and scene understanding.

By Xin Zhou, Dingkang Liang, Xiwu Chen, Feiyang Tan, Dingyuan Zhang, Hengshuang Zhao, Xiang Bai
arXiv Machine Learning
Oct 9

One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts

The paper introduces reViT, a recurrent vision transformer that uses a single Transformer block repeatedly to achieve the accuracy of a full-depth encoder while keeping inference FLOPs comparable. It achieves depth‑specific transformations by expressing each block’s feed‑forward network as a convex combination of a small shared expert bank, with a continuous depth coordinate guiding the mixture. Experiments on ImageNet‑1k and DINOv2 distillation show that reViT matches or surpasses state‑of‑the‑art models, reduces stored parameters by about 70%, and supports elastic‑depth deployment from a single checkpoint.

By Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
arXiv Computer Vision
Oct 5

GeoScaffold: Learning Compact Geometric Latents via Reconstruction for Efficient Vision-Language Navigation

GeoScaffold introduces a geometric supervision framework that embeds 3D geometry into vision‑language navigation policies during training, eliminating the need for depth sensors or 3D encoders at inference. The method learns a compact depth tokenizer from training trajectories, then fine‑tunes the policy with geometry query tokens that reconstruct depth, connectivity, and traversability, turning these states into compact geometric latents for action decoding. After training, the tokenizer, generators, and reconstruction heads are discarded, leaving a lightweight backbone that outperforms vision‑only navigators on continuous VLN benchmarks.

By Yixuan Jiang, Wentong Li, An Liu, Zihao Xin, Fulin Tang, Cong Leng, Yang Gao, Jian Cheng