arXiv Computer Vision
GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Machine Learning
Sep 24
Less Language, More Latents: Annotation-Efficient VLAs for Driving
The paper introduces Latent Action Driving Annotations (LADA), a three‑stage pipeline that converts large amounts of unlabelled observation‑trajectory data into a language‑conditioned driving model. First, a latent action model with a vector‑quantised bottleneck learns a compact codebook of vehicle intents. Then, a small set of language‑annotated examples trains a vision‑language translator to map observations and instructions into this codebook, and finally a VLA is trained on observation‑latent‑action pairs across the full corpus. Using less than 5% of language annotations, LADA attains a Driving Score of 87.98 and a Success Rate of 70.46% on Bench2Drive, matching or surpassing fully supervised baselines.
arXiv Computer Vision
Sep 22
Relationally Grounded Latent World Models for Autonomous Driving
Relationally Grounded Latent World Models for Autonomous Driving proposes using traffic scene graphs as privileged semantic supervision for latent world representations. The approach builds actor‑centric scene graphs from nuScenes 3D annotations, encodes their relational structure with a frozen text embedding model, and aligns visual latent representations to this semantic target during training. At inference the supervision branch is removed, requiring no scene graphs or 3D annotations and adding no extra computation, while achieving a 5.9% reduction in average trajectory L2 error and a 52.4% drop in collision rate compared to the LAW baseline.
arXiv AI
Oct 1
ChronoGraph: Functional 4D Scene Graphs with Vision-Language Models for Interaction Understanding and Grounded Planning
arXiv:2609.39665v1 Announce Type: new Abstract: Embodied agents must determine where to act, anticipate the resulting scene changes, and interpret observed outcomes to guide subsequent actions. This...
arXiv AI
Jul 21
Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering
arXiv:2510. 01483v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) demonstrate strong image-level scene understanding, but reasoning over long egocentric video remains costly: because VLMs maintain no persistent memory or explicit spatial representation, all sampled frames must be re-processed for every new query.
arXiv AI
Oct 15
DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving
DriveWLA is a unified World‑Language‑Action model that integrates semantic understanding, future scene dynamics, and trajectory generation for autonomous driving. It employs a Mixture‑of‑Transformers architecture with three experts—World, Language, and Action—connected by Cross‑Expert Attention to align semantic and dynamic representations. After reinforcement learning refinement, DriveWLA outperforms existing VLM‑based and world‑model‑based planners on NAVSIM benchmarks and generalizes well to other datasets without fine‑tuning.
arXiv Computer Vision
Sep 16
NeuroSymbEAD: A Large Scale Neuro-Symbolic Caption Dataset for Omni-Directional Embodied Autonomous Driving
NeuroSymbEAD is a large‑scale neuro‑symbolic caption dataset that builds an ego‑centric knowledge graph of static and dynamic objects on the KITTI‑360 dataset, annotating classes, categories, heading directions, orientations, and distances from the ego‑vehicle. The dataset generates multilevel textual captions that serve as a lightweight representation of an ego‑centric scene map, enabling outdoor scene‑map reconstruction, visual recognition, and object grounding. Baselines for driving common sense and traffic/scene understanding are established, and the dataset is benchmarked using pre‑trained grounding and learned auto‑regressive captioning networks to support vision‑language and foundation models for traffic‑scene explanation, 3D reasoning, and interpretable autonomous‑driving perception.