arXiv Computer Vision

Aggregating Neighbor Embedding Projection and Rank-Based Manifold Learning for Image Retrieval

The paper introduces a new image retrieval framework that merges neighbor embedding projections with rank-based manifold learning via rank aggregation. It uses UMAP to create low‑dimensional feature representations and combines ranked lists from UMAP and rank‑based re‑ranking methods using the Borda Count strategy. Experiments on public datasets with ResNet152, Swin Transformer, and DINOv2 features show that this combined approach improves retrieval performance, especially in scenarios where baseline representations have low precision.

arXiv Machine Learning
Sep 1

Context-Aware Interpretable Representations for Retrieval and Graph Convolutional Network Classification

The paper introduces an unsupervised framework that merges manifold learning with rank‑based interpretable graph embeddings to address the Geometric and Interpretability Gaps in visual representation learning. By first analyzing contextual information on the dataset manifold and then producing sparse, self‑explainable embeddings, the method achieves dimensionality reduction while preserving or improving performance in image retrieval and semi‑supervised Graph Convolutional Network classification. Experiments across varied datasets confirm that these context‑aware representations maintain high downstream effectiveness.

By Thiago C\'esar Castilho Almeida, Gustavo Rosseto Let\'icio, Vinicius Atsushi Sato Kawai, Daniel Carlos Guimar\~aes Pedronette
arXiv Machine Learning
Sep 1

Effective Graph and Rank-based Contextual Embeddings for Textual and Multimedia Data

arXiv:2608.29001v1 Announce Type: new Abstract: In a data-driven world, efficiently organizing and mapping relationships between objects is crucial. Graphs are powerful tools for modeling these conne...

By Thiago C\'esar Castilho Almeida, Gustavo Rosseto Let\'icio, Lucas Pascotti Valem, Andr\'e Freitas, Daniel Carlos Guimar\~aes Pedronette
arXiv AI
Sep 4

CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation

CORE improves compositional reasoning in multimodal language models by distilling a cross‑attentive reranker’s fine‑grained judgments into the embedding model. It generates candidate lists across five compositional matching levels and trains with a Rank‑KL objective to replicate the reranker’s ranking. Experiments on COLA, SUGARCREPE++, and NEGBENCH show CORE‑RERANKER‑8B outperforms Jina‑Reranker by 10.7 points, while CORE‑EMBED‑8B achieves the best overall average among evaluated embeddings, with gains also transferring to the MCMR benchmark without harming COCO or Flickr30K retrieval.

By Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Chu Liu, Pengjun Xie, Yilun Zhao, Shu Wu
arXiv Computer Vision
Aug 25

Learning Sample-wise Rank-aware Interpolation Weights for Composed Visual Data Retrieval

The paper introduces SRAIN, a framework that learns sample‑wise, rank‑aware interpolation weights for composed visual data retrieval. Instead of relying on complex multimodal large language models, SRAIN uses simple linear interpolation in embedding space, dynamically predicting query‑specific weights through batch‑wise rank‑aware estimation and a compact memory bank for hard negatives. This approach achieves state‑of‑the‑art performance on composed video retrieval and competitive results on composed image retrieval while significantly reducing query‑time latency.

By Boseung Jeong, Taegyu Park, Donghyeon Kwon, Hyunsouk Cho, Suha Kwak
arXiv AI
Jun 17

Graph Neural Networks for Semi-Supervised Image Classification with Multi-Feature Aggregation

arXiv:2606. 17406v1 Announce Type: cross Abstract: Feature extraction involves the identification and extraction of salient characteristics or patterns, including edges, textures, shapes, and color attributes.

By Marina Chagas Bulach Gapski, Vinicius Atsushi Sato Kawai, Gustavo Rosseto Leticio, Lucas Pascotti Valem, Daniel Carlos Guimar\~aes Pedronette, Mohand Said Allili
arXiv Computer Vision
6d ago

Through the PRISm: Importance-Aware Scene Graphs for Semantic Image Retrieval

The paper introduces PRISm, an image-to-image retrieval framework that prioritizes important elements in scene graphs for semantic image search. PRISm uses an Importance Prediction Module, trained with caption-derived relevance, to prune irrelevant objects and relations, and an Edge-Aware Contextual GNN to fuse the refined graph with global visual features into a single embedding. Experiments on PSG and Flickr show that PRISm outperforms strong baselines in MAP@1 and top‑1 metrics while adding only 33 ms per query beyond scene‑graph generation.

By Dimitrios Georgoulopoulos, Nikolaos Chaidos, Angeliki Dimitriou, Giorgos Stamou
arXiv Machine Learning
Jun 4

On Out-of-sample Embedding in UMAP

arXiv:2606. 04451v1 Announce Type: new Abstract: Neighbor embedding algorithms reveal correlations in high-dimensional data by constructing an equivalent graph representation in a lower-dimensional space.

By Mohammad Tariqul Islam, Jason W. Fleischer
arXiv AI
Sep 2

When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection

The paper introduces Inverted Contrastive Learning for Unsupervised Feature Selection (ICLFS), a method that treats each feature as a sample by inverting the data matrix and applies a contrastive learning framework to learn consistent representations across masked positive views and a shuffled negative view. Feature saliency is derived from the magnitude of projector‑space embeddings, and a Laplacian‑Gated Ranking Correction step refines the ranking by reducing local redundancy. Experiments on 12 benchmark datasets show that ICLFS achieves the best clustering accuracy on 10 datasets compared to both classical and neural baselines, demonstrating the effectiveness of feature‑wise contrastive consistency for unsupervised feature selection.

By Utsab Ghosh, Roshni Chakraborty
arXiv Machine Learning
6d ago

Domain-specific language alignment of multimodal embeddings towards explainable predictions

The paper introduces a framework that aligns multimodal embeddings with language by using captioning functions on interpretable auxiliary data, a soft contrastive loss, and a rank-based concept retrieval task. Applied to Earth observation embeddings, the method improves species distribution modelling, achieving state‑of‑the‑art performance on most metrics and better concept retrieval than prior vision‑language models. The aligned concepts enable recovery of species‑habitat associations and allow domain experts to interpret model predictions in natural language.

By Gabriel\.e Tij\=unaityt\.e, Thijs L. van der Plas, Mohamed-Bachir Belaid, Rob Knapen, Maciej J. Soja, Ioannis N. Athanasiadis