arXiv Computer Vision
Sep 3

Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker

The paper introduces OpenRef, a benchmark for Referring Expression Comprehension (REC) designed for open‑world scenarios. OpenRef expands beyond simple settings by including diverse visual domains, variable target counts (multi‑target and none‑target), and a rich vocabulary with proper nouns, polysemous words, and ordinal terms. It also proposes new evaluation metrics—F1 for grounding accuracy and N3R for negative expression rejection—and presents a training‑free Multi‑task Consistency Checker (MCC) that improves model performance with a single click.

By Zongjian Wu, Lei Zhang
arXiv Computer Vision
4d ago

Grounded Human-Attributed Description and Activity Recognition in Videos

The paper introduces Grounded Human-Attributed Description and Activity Recognition (GHADAR), a task that requires describing the characteristics and activities of every person in a video given their locations. It presents a new dataset derived from AVA-Actions with open‑set captions, a method that constrains cross‑attention masks in vision‑language models to better use grounding information, and a holistic VLM‑based evaluation schema that compares concepts directly between video and predictions. Experiments show the proposed method surpasses state‑of‑the‑art VLMs and highlight limitations of existing evaluation metrics.

By Jay N. Paranjape, Jurijs Nazarovs, Rohan Sarkar, Yue Guo, Lipeng Ke, Sankar Venkataraman, Vishal M. Patel, Nataraj Jammalamadaka