SCOUT: Sim-to-Real Text-Based Person Retrieval by Embedding-Space Prediction over Frozen Video Features

SCOUT is a frozen‑encoder approach for sim‑to‑real text‑based person retrieval that predicts cross‑modal embeddings instead of fine‑tuning cross‑encoders. It uses a trainable predictor to map patch tokens from a frozen video encoder (V‑JEPA) into the embedding space of a frozen text encoder (EmbeddingGemma), guided by a bidirectional InfoNCE objective. The method achieves state‑of‑the‑art results on the AI City Challenge 2026 Track 4, with a full retrieve‑fuse‑rerank pipeline reaching 84.25 mAP@10 and a single frozen model alone scoring 60.63, while training costs are modest (≈95 GPU‑hours).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

• llms
• rag
• robotics
• fine-tuning
• multimodal
• benchmarks
• safety