arXiv Computation and Language
Sep 10

Vague2Detect: Handling Ambiguous Prompts in Knowledge-Based Open-World Detection

Vague2Detect is a hybrid pipeline that improves object detection for ambiguous prompts by combining a fine‑tuned Sentence‑BERT to retrieve candidates from a structured household knowledge base, YOLO‑World to verify their presence in images, and a GPT‑3.5‑turbo fallback to generate new candidate descriptions when prompts fall outside the knowledge base. On a benchmark of household scenes, Vague2Detect raises the vague prompt success rate from 32% (YOLO‑World alone) to 61% with high precision, and up to 85% when the GPT fallback is used.

By Ibrohimjon Muminov (Dongguk University, Seoul, South Korea), Jihie Kim (Dongguk University, Seoul, South Korea)