The paper introduces ORMOT, a new task that extends Referring Multi‑Object Tracking to omnidirectional 360° imagery, ensuring full scene context for language‑guided tracking. It presents ORSet, a dataset of 27 omnidirectional scenes with 848 language descriptions and 3,401 annotated objects, and introduces ORTrack, an LVLM‑driven framework that performs zero‑shot detection and robust cross‑frame association. Experiments on ORSet show that ORTrack achieves state‑of‑the‑art performance, establishing a strong baseline for future research.
By Zihan Zhou, Sijia Chen, Yanqiu Yu, En Yu, Wenbing Tao
arXiv:2609.09396v1 Announce Type: new
Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...
By Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain, Lap Fung Chan, John Suchanek, Yu Wang, Varun Praveen, Tomasz Kornuta, Vidya Nariyambut Murali
The paper introduces Referring Multi-Object Tracking (RMOT), a task that uses natural language expressions to guide the prediction of multiple objects over time, addressing limitations in previous work that lacked dynamic language expressiveness. It presents the Refer-KITTI-V2 benchmark, which includes 9,758 semi-automatically generated language prompts for diverse object dynamics. The authors also propose TempRMOT, a Transformer-based framework with a Temporal Enhancement Module that achieves state‑of‑the‑art results on both Refer‑KITTI and Refer‑KITTI‑V2.
By Yani Zhang, Dongming Wu, Wencheng Han, Xingping Dong, Shengcai Liao, Bo Du
ARGOS is a new benchmark and agent framework for multi‑camera person search that transforms the task from one‑shot retrieval into interactive reasoning using partial witness clues. The framework requires agents to plan questions, use spatial or temporal tools, and interpret ambiguous natural‑language responses within a limited turn budget, leveraging a Spatio‑Temporal Topology Graph that encodes camera connectivity and transition times. The benchmark includes 2,691 tasks across 14 real‑world scenarios, divided into semantic, spatial, and temporal tracks, and introduces Turn‑Weighted Success (TWS) as a metric that jointly measures correctness and turn efficiency, with current best agents achieving TWS scores of 0.383 and 0.590 on the spatial and temporal tracks respectively.
By Myungchul Kim, Kwanyong Park, Junmo Kim, In So Kweon
YesTrack introduces a two‑stage referring multi‑object tracking approach that treats the task as a discriminative Yes/No verification problem, directly using multimodal large language models (MLLMs) without generating captions. It adds lightweight temporal consistency constraints—Temporal Confidence Prior (TCP) and Temporal Reference Propagation (TRP)—to improve reliability and efficiency. The method also extends to generic multi‑object tracking as YesTrack‑MOT, achieving state‑of‑the‑art performance on Refer‑KITTI datasets while remaining efficient even with the smallest Qwen3‑VL variant.
By Quansheng Hu, Qin Sun, Qiansen Dai, Jin Ding, Wan Zhang, Xue Zhou, Jianxiao Zou
CRTrack-Cap is a new benchmark for cross‑view dense video object captioning that links tracklets from multiple camera views into unified global trajectories and generates a single caption per trajectory. The dataset, derived from CRTrack, contains 425 global trajectories, 838 local tracklets across 13 scenes, and canonical captions for 335 identities. A reference framework combining VECA and CEG achieves strong CV‑CHOTA scores, offering a baseline for consistent multi‑camera person description.
By Xiangqun Zhang, Likai Wang, Zekun Qian, Rui Zhou, Ruize Han, Wei Feng