Hugging Face Trending Papers

CR-Refiner: An Object-Centric Optimal Transport Reranker for Edit-Conditioned 3D Scene Retrieval

Edit-conditioned 3D scene retrieval pairs a reference 3D room with a natural-language modification and retrieves rooms from a corpus that satisfy the edit. Three lines of prior work each fall short on this task.

arXiv AI
Sep 4

CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation

CORE improves compositional reasoning in multimodal language models by distilling a cross‑attentive reranker’s fine‑grained judgments into the embedding model. It generates candidate lists across five compositional matching levels and trains with a Rank‑KL objective to replicate the reranker’s ranking. Experiments on COLA, SUGARCREPE++, and NEGBENCH show CORE‑RERANKER‑8B outperforms Jina‑Reranker by 10.7 points, while CORE‑EMBED‑8B achieves the best overall average among evaluated embeddings, with gains also transferring to the MCMR benchmark without harming COCO or Flickr30K retrieval.

By Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Chu Liu, Pengjun Xie, Yilun Zhao, Shu Wu
arXiv AI
Jun 9

Generative Reasoning Re-ranker

arXiv:2602. 07774v5 Announce Type: replace-cross Abstract: Recent studies increasingly explore Large Language Models (LLMs) as a new paradigm for recommendation systems due to their scalability and world knowledge.

By Mingfu Liang, Yufei Li, Jay Xu, Kavosh Asadi, Xi Liu, Shuo Gu, Kaushik Rangadurai, Frank Shyu, Shuaiwen Wang, Song Yang, Zhijing Li, Jiang Liu, Mengying Sun, Fei Tian, Xiaohan Wei, Chonglin Sun, Jacob Tao, Shike Mei, Wenlin Chen, Santanu Kolay, Sandeep Pandey, Hamed Firooz, Luke Simon
arXiv AI
Sep 1

ScenePilot: Grow-and-Repair Policy for Text-Driven 3D Indoor Scene Generation

ScenePilot introduces a retrieval‑augmented Grow‑and‑Repair framework for text‑driven 3D indoor scene generation. It uses a Hierarchical Retrieval‑Augmented Planning module to fetch room, group, and anchor layout priors, then incrementally inserts object groups with a base generator, while a Reinforcement Multimodal Repair module performs lightweight local corrections after each insertion and a final global repair. The approach is trained on a new SceneReverse‑17k dataset of perturbed scenes, enabling the policy to predict structured move‑rotate‑scale actions from rendered views, scene state, retrieved priors, and edit history, thereby improving physical plausibility, functional coherence, and controllability without heavy full‑scene optimization.

By Jiawei Zhang, Hongsong Wang, Pan Zhou
arXiv Computer Vision
Sep 24

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

The paper introduces RSMEB, a unified benchmark for remote‑sensing multimodal retrieval that evaluates both cross‑modal and interleaved retrieval across 21 tasks under a single ranking protocol. It also presents VLM2GeoVec, an instruction‑conditioned single‑encoder model that embeds image, text, bounding‑box, and geo‑coordinate tokens into one sequence and achieves state‑of‑the‑art performance on region‑caption, referring‑expression, and semantic geo‑aware retrieval while remaining competitive on conventional tasks. The authors provide code, checkpoints, and data on GitHub to facilitate reproducibility.

By Emanuel S\'anchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan, Yonghao Xu, Michael Felsberg
Hugging Face Trending Papers
Aug 3

SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models

Vision-language models (VLMs) achieve strong semantic understanding but remain unreliable in metric spatial reasoning, particularly when queries require comparing multiple instances of the same object category. We study this problem through the Closest-Instance Distance Query (CIDQ), where a model must identify the nearest visible candidate to a unique reference object and estimate their gravity-aligned floor-plane distance.

arXiv Computer Vision
Sep 3

GeoStore: Finding Small Storefronts in Large Scenes -- A Fine-Grained POI Localization Benchmark with Global-to-Local Asymmetric Matching

GeoStore is a new benchmark for fine‑grained point‑of‑interest (POI) localization that matches close‑up storefront photos against large geo‑tagged street‑view images, a task distinct from traditional visual place recognition. The paper shows that global‑descriptor methods designed for symmetric matching perform poorly on this asymmetric problem, and introduces GLAM, a Global‑to‑Local Asymmetric Matching approach that combines a global retrieval anchor with a lightweight local re‑ranking using pooled region tokens. GLAM achieves higher Recall@1/5/10 and mAP than strong baselines while using far fewer re‑ranking features and significantly lower per‑pair matching cost.

By Lu Han, Xiting Sun, Hao Wang, Zhiqiang Cao, Ruihuan Du, Ziquan Zeng, Chunlong Lv