arXiv Computer Vision By Tairan Fu, Francisco Javier Santos-Mart\'in, Javier Conde, Elena Merino-G\'omez, Pedro Reviriego

Lost in Motion: Vision Language Models Fail the Dynamic Gauges Test

Read the original on arXiv Computer Vision →

The paper titled "Lost in Motion: Vision Language Models Fail the Dynamic Gauges Test" examines how current Vision‑Language Models (VLMs) perform on real‑time gauge reading tasks in industrial settings. Using a new dataset of video sequences featuring circular, linear, and Vernier gauges under varied motion, the study tests state‑of‑the‑art models from GPT‑5 and Gemini 3 families. Results show that these VLMs struggle to accurately interpret needle trajectories and scale semantics, falling short of the traceability and reliability required for safety‑critical monitoring and thus not meeting IEEE and ISO standards for trustworthy synthetic instruments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Aug 18

OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models

arXiv:2602. 18094v2 Announce Type: replace-cross Abstract: Existing Visual-Language Models (VLMs) have achieved significant progress by being trained on massive-scale datasets, typically under the assumption that data are independent and identically distributed (IID).

By Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen
arXiv Computer Vision
5d ago

World Models' Last Exam in Physics

arXiv:2610.08791v1 Announce Type: new Abstract: Video world models can produce visually convincing yet physically inconsistent sequences, raising concerns about their reliability for prediction and p...

By Mingju Gao, Qingle Liu, Yuzhao Peng, Xinjie Lin, Ziming Qin, Zheng Jiang, Wenyi Li, Calvin Xiao, Youjie Zheng, Kaisen Yang, Qinhuai Na
arXiv AI
Oct 1

DrivingBench: Can Vision-Language Models Drive a Toyota Corolla?

DrivingBench is the first benchmark that tests general‑purpose vision‑language models on the task of driving a real Toyota Corolla around a parking‑lot cone course. The models receive live camera frames and issue steering and velocity commands, with inference latency counted as part of the challenge. In tests, only GPT‑6 Astra completed the course, while other models showed limited progress or failed to pass half the course.

By Aditya Ramabadran, Simon Mahns, Tobias Gessler
arXiv AI
Sep 23

RiverVLN: Phase-Grounded Temporal Vision--Language Navigation for Unmanned Surface Vehicles

RiverVLN introduces the first benchmark for long‑horizon vision‑language navigation (VLN) of unmanned surface vehicles (USVs) in continuous riverine motion. The PGT‑NAV framework converts navigation instructions into an ordered sequence of visually verifiable semantic phases, maintaining an active phase online through grounded visual and motion evidence. This phase‑grounded approach reduces recursive position and heading drift, achieving a 0.79 success rate in Unity‑ROS closed‑loop tests and demonstrating transfer to real‑world USV deployment.

By Jieling Wu, Yuehao Huang, Jiajun Lv, Tao Huang, Yong Liu, Weiwei Liu
arXiv AI
Sep 4

LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving

LightEMMA is a longitudinal evaluation framework that tests the autonomous driving performance of vision‑language models (VLMs) without fine‑tuning or prompt engineering. Using this protocol, the authors evaluated 15 VLMs from five major families on the nuScenes prediction benchmark and found that larger, more capable models do not consistently outperform earlier generations. The study identifies common failure modes such as overreliance on historical actions and difficulty reconciling conflicting visual cues, underscoring the need for domain‑specific adaptation to enhance VLM safety in autonomous driving.

By Zhijie Qiao, Haowei Li, Zhong Cao, Henry X. Liu