RankGround is a two‑stage framework for GUI grounding that uses a single Vision‑Language Model call per query. It introduces GroundRanker, a lightweight multimodal reranker that selects the most promising crop from a dense candidate set, trained with a two‑stage curriculum on ranking supervision data derived from existing grounding datasets. Experiments show RankGround outperforms strong baselines, achieving 1.4× faster inference and a 5.5% average improvement in localization accuracy over the second‑best method across all backbones and screen scales.
arXiv:2608. 03270v1 Announce Type: cross Abstract: GUI grounding maps natural-language instructions to click locations and is essential for reliable GUI agents.
arXiv:2608.03270v2 Announce Type: replace Abstract: GUI grounding maps natural-language instructions to click locations and is essential for reliable GUI agents. The task remains difficult on high-re...
arXiv:2610.33645v1 Announce Type: cross Abstract: GUI grounding requires locating small controls in cluttered screenshots and identifying which control a natural-language instruction refers to. Cropp...
The paper introduces MHVF, a training‑free framework for GUI grounding that uses multi‑hypothesis exploration and local visual verification with a single pretrained vision‑language model. MHVF generates spatial hypotheses from full‑screen predictions, refines them through zoomed re‑localization, and verifies candidate agreement with the instruction to resolve ambiguity. Experiments on eight benchmarks with five models show significant accuracy gains, notably improving UI‑Venus‑Ground‑7B by 4.3 pp on Mind2Web and raising Qwen2.5‑VL‑7B’s ScreenSpot‑Pro accuracy from 23.6 % to 53.6 %.
arXiv:2603.16932v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) typically process images at a native high-resolution, forcing a trade-off between accuracy and computational ef...
AgriScope is a unified pixel‑grounded multimodal framework designed for agricultural image understanding. It supports image‑level, region‑level, and pixel‑level tasks such as grounded caption generation, referring expression segmentation, and multi‑turn multimodal interaction. The authors also introduce AgriGround, a large‑scale dataset with over 500K images and 11M instruction‑following samples, created via an automatic annotation pipeline that combines caption generation, phrase‑level grounding, segmentation mask creation, and instruction synthesis.
IVSGround introduces a lightweight view selector that learns to choose the most informative camera views for vision‑language model (VLM) based 3D visual grounding, replacing heuristic view selection. The selector is trained via a two‑stage rejection sampling process that uses feedback from a reasoning VLM to generate supervision signals. Experiments on ScanRefer and NR3D demonstrate that IVSGround consistently improves grounding accuracy over existing zero‑shot pipelines, underscoring the importance of selecting where to look for effective 3D visual grounding.
AgroGround is a large‑scale dataset that links plant disease and agricultural target labels to image regions, enabling models to both recognize and localize these targets. It was created by automatically converting eight agricultural VQA datasets into 794,850 grounded instruction examples, including healthy images for negative supervision. Fine‑tuning a shared vision‑language model on these instructions improves recognition, joint accuracy, and healthy‑image abstention, outperforming its annotation teacher on both the AgroGround benchmark and the external PlantSeg test set.
arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...
Jag is a method that transforms a compact pretrained multimodal model into a direct box predictor for visual grounding. It adds a lightweight regression head that, conditioned on the image and referring expression, outputs all four box coordinates in a single forward pass without sequential decoding. Geometric supervision jointly fine‑tunes the language backbone, visual merger, and regression head, leading to improved accuracy and lower latency compared to existing specialized grounding models.
AgriScope is a unified pixel‑grounded multimodal framework designed for agricultural image understanding, supporting image‑level, region‑level, and pixel‑level tasks such as grounded caption generation, referring expression segmentation, and multi‑turn multimodal interaction. It incorporates biologically specialized semantic representations with dense spatial grounding through biological‑semantic encoding, dense spatial representations, and pixel decoding. The authors also introduce AgriGround, a large‑scale dataset of over 500K images and 11M instruction‑following samples, created via an automatic annotation pipeline that combines caption generation, phrase‑level grounding, segmentation mask generation, and task‑oriented instruction synthesis to provide densely grounded supervision for agricultural vision‑language learning.
arXiv:2511. 00810v4 Announce Type: replace-cross Abstract: Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regions on the screen.