Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,124 stories · RSS feed

arXiv Computer Vision
6d ago

MLLM Frame Subset Ensembling for Audio-Visual Video QA and MLLM-based Reranking for Ad-hoc Video Search in TRECVID 2025

The paper describes the CERTH-ITI team’s approach to the TRECVID 2025 Ad-hoc Video Search (AVS) and Video Question Answering (VQA) tracks. For AVS, a two-stage framework uses foundation vision‑language models for initial retrieval and a Multimodal‑LLM for reranking, improving contextual matching without training. In VQA, an audio‑visual MLLM is fine‑tuned and enhanced with inference‑time scaling; answer generation employs majority voting over responses from random frame subsets, while multiple‑choice responses use mean‑pooling logits, yielding top performance on VQA metrics.

By Andreas Goulas, Damianos Galanopoulos, Ioannis Patras, Vasileios Mezaris
arXiv Computer Vision
6d ago

TAN: A Size- and Form-Adaptive Masked-Convolution Encoder for Tissue Graphs

TAN (Tissue‑Aware Network) is a lightweight convolutional encoder that processes tissue‑conforming superpixels in their native shape, avoiding background leakage from padding or masking. It achieves accuracy on the BACH histology dataset comparable to a 303‑million‑parameter foundation model while using only 0.41 M parameters and 1,190× fewer FLOPs, and its embeddings are invariant to the choice of fill value. On the BRACS dataset, TAN performs close to a ResNet18 trained under the same protocol and outperforms standard masked‑crop processing with less than half the computational cost.

By Jan Herschel, Anselm von Gladiss
arXiv Computer Vision
6d ago

GazeCLIP: Gaze-Guided CLIP with Adaptive-Enhanced Fine-Grained Language Prompt for Deepfake Attribution and Detection

The paper introduces GazeCLIP, a gaze‑guided CLIP framework that uses adaptive‑enhanced fine‑grained language prompts for deepfake attribution and detection (DFAD). It presents a new benchmark evaluating DFAD on novel generative methods such as diffusion and flow models, and proposes a gaze‑aware image encoder (GIE) that fuses gaze prompts with image embeddings to capture global forgery patterns. A language refinement encoder (LRE) further refines language embeddings for precise vision‑language matching, achieving state‑of‑the‑art performance with 6.56% higher accuracy and 5.32% higher AUC on the benchmark.

By Yaning Zhang, Linlin Shen, Zitong Yu, Chunjie Ma, Zan Gao
arXiv Computer Vision
6d ago

Through the PRISm: Importance-Aware Scene Graphs for Semantic Image Retrieval

The paper introduces PRISm, an image-to-image retrieval framework that prioritizes important elements in scene graphs for semantic image search. PRISm uses an Importance Prediction Module, trained with caption-derived relevance, to prune irrelevant objects and relations, and an Edge-Aware Contextual GNN to fuse the refined graph with global visual features into a single embedding. Experiments on PSG and Flickr show that PRISm outperforms strong baselines in MAP@1 and top‑1 metrics while adding only 33 ms per query beyond scene‑graph generation.

By Dimitrios Georgoulopoulos, Nikolaos Chaidos, Angeliki Dimitriou, Giorgos Stamou
arXiv AI
6d ago

EKLAV: Elicited Key CoT Learning via Answer only superVision

EKLAV is a method that uses a teacher’s chain‑of‑thought (CoT) reasoning as privileged input while supervising only the final answer. By strategically placing parts of the trace in the prompt, EKLAV improves ranking performance on BRIGHT passage reranking and table‑reranking benchmarks, outperforming full‑trace distillation and answer‑only training. The approach reduces training compute and inference time, and the generated traces show more efficient reasoning patterns.

By Adarsh Singh, Aman Singh, Kushal Raj Bhandari, Soham Dan, Anoop Kumar, Vivek Gupta
arXiv Computation and Language
6d ago

Evidence-Gated Bilingual RAG for Reliable Consumer Banking Search

arXiv:2610.22652v1 Announce Type: new Abstract: Consumers often describe what they need rather than use the terminology found on a banking website. For example, a natural-language request ("a card fo...

By Ahmed Mohamed Mahmoud Rayyan, Rana Mahmoud Mohammed, Aya Abdallah Abdelmonem, Mohamed Khaled Goda Othman, Rahatara Ferdousi
arXiv Machine Learning
6d ago

Deep LLM layers expose passage-level context in EEG during silent reading

The study investigates how information about words and passage context is represented across the layers of the Llama‑3.1‑8B language model during silent reading, using EEG recordings. Decoders trained on each layer show that word‑level decodability peaks in early layers and diminishes toward the final layer, while a context‑tracking component increases with depth. Even a single 0.5‑second EEG window can capture passage‑level information, and control experiments confirm that this context tracking is driven by the text rather than by word decoding or session artifacts.

By Ingo Marquardt, Anthilia Alchanat, Priyanka Jain
arXiv AI
6d ago

Frozen Models, Evolving Expertise: Model-Agnostic Learning from Deployment Experience for Multimodal Medical AI

The paper introduces a model‑agnostic framework that lets frozen large language models (LLMs) and vision‑language models (VLMs) learn from deployment experience without retraining. It does so by adding three external expertise components—a Skill for guiding reasoning and tool use, a Knowledge Memory for storing reliable facts, and a Multimodal Knowledge Base for visual examples—and by using a validation strategy that updates only when new cases improve performance without harming past performance. Experiments on six benchmarks with both open‑weight and closed‑source models show up to a 34.2% performance gain on medical tasks, generalization to unseen cases, and transferability to other models and non‑medical domains.

By Yexiao He, Yucheng Tang, Pengfei Guo, Yufan He, Andriy Myronenko, Can Zhao, Ang Li, Daguang Xu, Dong Yang