The deployment of Vision-Language Models (VLMs) in autonomous driving (AD) systems is constrained by on-board computing power, restricting vehicles to small VLMs (SVLMs) with limited perception and reasoning capabilities. Infrastructure-assisted AD alleviates this resource constraint by enabling collaboration with large VLMs (LVLMs) at edge servers.
Driving world-action models improve planning by coupling multimodal reasoning with future prediction, but their growing inference cost increasingly conflicts with the real-time latency requirements of...
LightEMMA is a longitudinal evaluation framework that tests the autonomous driving performance of vision‑language models (VLMs) without fine‑tuning or prompt engineering. Using this protocol, the authors evaluated 15 VLMs from five major families on the nuScenes prediction benchmark and found that larger, more capable models do not consistently outperform earlier generations. The study identifies common failure modes such as overreliance on historical actions and difficulty reconciling conflicting visual cues, underscoring the need for domain‑specific adaptation to enhance VLM safety in autonomous driving.
SlackDrive is a pre‑inference compute allocator that dynamically selects the compute budget for each driving control step by reusing the realized latency from previous inferences. By profiling a small set of discrete budgets once, it estimates the current compute state online and chooses the highest‑utility budget that stays within the admissible latency envelope. On the NAVSIM v2 benchmark with DriveDreamer‑Policy, SlackDrive boosts latency‑constrained EPDMS performance by 21.7% compared to the best baseline, while full‑budget and token‑pruning approaches exceed the latency limits under runtime contention.
The paper introduces CloudEdgeVLA, a cloud‑edge policy for Vision‑Language‑Action models that treats temporal misalignment as a representation‑learning problem. It encodes delayed observations into slowly varying task features on the cloud while a lightweight edge head fuses the latest cloud feature with current local vision. Experiments on four LIBERO suites show that CloudEdgeVLA retains 63.8–78.0% success under a 40‑step delay window, far outperforming VLASH and single‑path baselines.
arXiv:2512. 05277v3 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.
The paper introduces FedQoS, an asynchronous federated learning framework designed for multimodal in‑cabin interaction in smart vehicles. It uses a two‑phase gating mechanism: a resource‑aware training gate that starts local learning only when sensing buffers and energy reserves meet safety thresholds, and a QoS‑aware transmission policy that gates uplink updates based on an efficiency score balancing model novelty, latency, and energy costs. Experiments on vehicular datasets show FedQoS achieves competitive personalized accuracy with only marginal loss compared to FedAvg, while reducing communication overhead by 76.7% and latency cost by 26.0%.
arXiv:2512. 05277v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) are increasingly deployed as the perception and reasoning backbone of autonomous agents acting in the wild, with autonomous driving (AD) being one of the most safety-critical instances.
arXiv:2608. 15636v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment.
arXiv:2607. 00283v1 Announce Type: cross Abstract: Autonomous vehicles must safely navigate complex environments where planning-critical agents may be hidden from view.
arXiv:2608. 12932v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models promise to bring end-to-end reasoning to autonomous driving, but their computational cost remains far too high for real-time control.
Vision-Language-Action (VLA) models promise to bring end-to-end reasoning to autonomous driving, but their computational cost remains far too high for real-time control. The core challenge is structural: VLA inference is not a single bottleneck but a cascade of four.
arXiv:2609.05516v1 Announce Type: cross Abstract: Unified perception enables autonomous driving systems to perform object detection, drivable-area segmentation, and lane segmentation within a single...