← Back to all news
arXiv Computer Vision By Abdarahmane Traor\'e, Andy Couturier, \'Eric Hervet

SCOUT: Sim-to-Real Text-Based Person Retrieval by Embedding-Space Prediction over Frozen Video Features

Read the original on arXiv Computer Vision →

SCOUT is a frozen‑encoder approach for sim‑to‑real text‑based person retrieval that predicts cross‑modal embeddings instead of fine‑tuning cross‑encoders. It uses a trainable predictor to map patch tokens from a frozen video encoder (V‑JEPA) into the embedding space of a frozen text encoder (EmbeddingGemma), guided by a bidirectional InfoNCE objective. The method achieves state‑of‑the‑art results on the AI City Challenge 2026 Track 4, with a full retrieve‑fuse‑rerank pipeline reaching 84.25 mAP@10 and a single frozen model alone scoring 60.63, while training costs are modest (≈95 GPU‑hours).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv Computation and Language
Oct 26

Semantically Rich, Metrically Affect-Blind: A Geometric Blind Spot in Modern Text Embedders

The paper uncovers a systematic mismatch in modern text embeddings: while sentiment can be extracted with high accuracy from frozen encoders, the embeddings’ cosine similarity metric barely reflects polarity differences. Across eleven models from 2021–2026, a linear probe achieves up to 91.3% accuracy, yet cosine similarity margins are only 0.003–0.064, leading to poor retrieval performance (e.g., Qwen3-Embedding-4B’s 64.3 MAP). Introducing a lightweight supervised‑contrastive head (Sentiment‑Injected Sentence Embeddings) dramatically improves probe accuracy, retrieval MAP, and polarity margins, but at the cost of downstream semantic similarity. "whyItMatters":"The findings reveal that widely used retrieval and clustering systems may ignore valuable sentiment information encoded in embeddings, impacting their effectiveness."

By Mahdi Naser Moghadasi
llmsragsafety
More like this →
arXiv Computer Vision
Oct 26

Synthetic-to-Real Text-Based Person Anomaly Search via Multi-Model Retrieval Fusion and Cross-Encoder Reranking

The paper introduces a multi‑model retrieval and reranking system for text‑based person anomaly search that bridges the synthetic‑to‑real domain gap. It adapts the Qwen3‑VL‑Embedding‑8B model to synthetic descriptions via contrastive learning and domain randomization, fuses scores from six Qwen3‑VL checkpoints and a frozen SigLIP2 model, and then reranks candidates with a Qwen3‑VL‑Reranker‑8B cross‑encoder. A synthetic distractor‑matched validation protocol is used to tune hyper‑parameters, yielding a 94.78% mAP and 90.85% R@1 on the AI City Challenge 2026 Track 4 test set, placing eighth on the public leaderboard.

By Eugene Denteh, Blessing Agyei Kyem, Andrews Danyo, Joshua Kofi Asamoah, Richard Dzinyela, Solomon Owusu-Ansah, Dickson G. Asiedu, Armstrong Aboah
ragmultimodalbenchmarks
More like this →
arXiv Computer Vision
Oct 9

VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval

VEDJE is a new video-text retrieval method that compresses video features into a reusable cache while keeping them distinct for each frame. It introduces feature-change prediction as an auxiliary training signal, enabling improved retrieval from the compressed cache without extra cost at query time. Experiments on MSR-VTT, MSVD, DiDeMo, and ActivityNet show that VEDJE boosts R@1 in both retrieval directions, achieving 59.8% text-to-video R@1 on MSR-VTT with a fine-tuned VideoCLIP-XL first stage and maintaining recall when the per-video cache is reduced to 12 KiB.

By Shahaf Wagner, Gabriele Serussi, Dan Ben Ami, Tomer Galanti, Chaim Baskin
diffusionfine-tuning
More like this →