Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,383 stories · RSS feed

arXiv Computer Vision
Aug 31

Training-Free Temporal Abstraction for General Video Understanding

The paper introduces STITCH, a training‑free method that partitions videos into semantically meaningful temporal chunks using a frozen video‑text backbone. By detecting changes in the embedding sequence of short video windows, STITCH produces reusable temporal abstractions that can be applied to multiple tasks such as event boundary detection, language‑based moment retrieval, and frame selection for vision‑language models. Experiments show that STITCH performs competitively with specialized methods while requiring no task‑specific training, especially when processing is limited to a few frames or tokens.

By Etienne Casanova, Sevan Brodjian, Pietro Perona
arXiv Computation and Language
Aug 31

Select, Label, Evaluate: Active Testing in NLP

The paper introduces Active Testing, a framework that selects the most informative test samples for annotation in NLP, aiming to reduce human effort while accurately estimating model performance. Experiments across 18 datasets and 4 embedding strategies show up to 95% annotation savings with less than 1% loss in performance estimation accuracy. The authors also propose an adaptive stopping criterion to determine the optimal number of samples without a predefined budget.

By Antonio Purificato, Maria Sofia Bucarelli, Andrea Bacciu, Fabrizio Silvestri, Amin Mantrach
arXiv Computer Vision
Aug 31

Thread-Efficient Decoding for Neural Texture Compression

Thread‑Efficient Decoding for Neural Texture Compression proposes a shared decoder MLP architecture trained with a gradual decoder freezing schedule and texture clustering to reduce GPU thread divergence by 25‑52%. The method preserves rendering quality while achieving up to 8.48× speedup on a Radeon RX 9070 XT compared to non‑shared baselines. Experiments on over 500 textures and multiple real rendering scenes validate the performance gains and provide comprehensive ablation studies.

By Janarbek Matai, Sho Ikeda, Lukasz Lipski, Takahiro Harada
arXiv Computer Vision
Aug 31

WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild

WilLaGS introduces a unified framework that enhances 3D Gaussian Splatting for in-the-wild scenes by learning a continuous global appearance manifold with a β‑VAE and generating dynamic Tri‑Plane features for spatially‑varying local illumination. It also incorporates a self‑supervised perceptual masking mechanism using a Teacher‑Student EMA architecture to suppress transient artifacts and identify inconsistent regions. Experiments on multiple datasets show that WilLaGS achieves state‑of‑the‑art reconstruction quality and novel view synthesis while preserving real‑time rendering efficiency.

By Yuhao Bai, Qianqiu Tan, Lilong Chen, Huanhuan Lv, Lijun Chen
arXiv Computation and Language
Aug 31

A Wolf in Sheep's Clothing: Targeted Routing Hijacking in Federated RAG

The paper exposes a new vulnerability in Federated Retrieval-Augmented Generation (FedRAG) called Routing Hijacking, where a malicious client forges its semantic profile to attract target queries despite lacking relevant data. Experiments across three FedRAG routing architectures show that this attack consistently misroutes queries, causing downstream failures such as missing evidence, poisoning, incorrect answers, hallucinations, and sycophantic behavior. The authors propose a trust‑aware post‑routing framework that reweights clients based on evidence feedback, which effectively suppresses persistent hijacking and transfers to a learned neural router.

By Junjie Mu, Qiongxiu Li
arXiv Computation and Language
Aug 31

Select, Don't Train: The Benefits of Modular Entity Disambiguation with LLM-Based Selection

The paper investigates modular entity disambiguation by separating candidate retrieval from entity selection. It compares sparse retrieval (BM25), Web KB search, and a dense retriever, all paired with large language model selectors. Results show that a training‑free BM25 retriever combined with an LLM selector achieves state‑of‑the‑art performance on the ZELDA benchmark, and the modular approach enables abstention when retrieval fails.

By Fina Polat, Daniel Daza, Pengyu Zhang, Klim Zaporojets, Paul Groth
arXiv Computation and Language
Aug 31

Embedding Models for Stance-Aware Argument Retrieval

The paper investigates how dense embedding models can be used for stance-aware argument retrieval, a task that requires both topic relevance and correct stance (support or attack) toward a claim. Experiments reveal that current models favor topical overlap and ignore stance, and that contrastive training to fix this bias leads to over-correction, where models focus too much on polarity keywords at the expense of topic relevance. To address this, the authors propose diagnostic word-ablation metrics and a data‑centric solution involving a balanced argument curriculum and LLM‑augmented stance‑inverted arguments, which helps powerful models learn deeper directional logic and improves stance‑aware retrieval performance.

By Angelo Sparacino, Francesca Toni, Adam Dejl
arXiv Computation and Language
Aug 31

Ladders in Chaos: When, How, (and Perhaps Why) Does Test-Time Scaling Improve LLM Machine Translation

The paper examines two test‑time scaling methods for large language models in machine translation: sequential sampling, where later attempts build on earlier ones, and parallel sampling, such as independent i.i.d. sampling with reranking. Sequential sampling shows a higher performance ceiling, offering a more diverse and effective set of translations, especially with limited sampling budgets. Human analysis reveals that while sequential sampling improves fluency and naturalness, it can reduce accuracy when the inference budget is large, and the authors attribute this effect to the model’s access to a larger target‑side context.

By Di Wu, Sergey Troshin, Christof Monz, Antske Fokkens, Vlad Niculae
arXiv Computer Vision
Aug 31

From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation

The paper introduces Distortion Extenders (DEX), learnable parameters that adapt vision foundation models to fisheye cameras by modeling distortion coefficients and correcting distributional shifts between fisheye and perspective images. DEX is applied to monocular depth estimation and open‑vocabulary segmentation across convolutional and Transformer architectures, consistently outperforming baselines on indoor and outdoor fisheye datasets. Additionally, DEX activations can be decoded to obtain distortion coefficients, aiding camera calibration.

By Rit Gangopadhyay, Alex Wong
arXiv Machine Learning
Aug 31

QGPINNs: A Physics-Informed Neural Network Framework for Nonlocal Differential Equations on Quantum Graphs

QGPINNs is a PyTorch-based physics-informed neural network framework for solving nonlocal differential equations on quantum graphs. It approximates the solution on each edge with a neural network and uses a unified graph‑based loss to enforce governing equations, initial, boundary, and vertex transmission conditions, including continuity, Kirchhoff‑Neumann, and Dirichlet conditions. The framework supports multi‑order fractional elliptic problems and time‑fractional evolution equations, incorporates graph‑adapted learning strategies such as soft/hard constraints, dynamic loss balancing, Fourier feature embeddings, and a learnable singularity‑capturing feature, and extends to inverse problems for identifying fractional orders and physical parameters from noisy data, as validated on benchmark and real‑world networks such as the IEEE 14‑bus system and an agricultural drainage network.

By Vaibhav Mehandiratta, Saket Ramchandra
arXiv Computer Vision
Aug 31

Abstract4D: A Large-Scale Dataset and Framework for Understanding the Visual Language of Abstract Art

Abstract4D is the largest dataset of abstract paintings, containing over 120,000 images with rich metadata and multi‑dimensional prompts that capture perceptual attributes such as form, color, texture, and composition. The dataset is annotated via a hybrid human–VLM pipeline to ensure quality and consistency. Using Abstract4D, the authors analyze the semantic structure of abstract art through large‑scale embedding visualization and establish benchmark tasks for classification, cross‑modal retrieval, and text‑to‑image generation to evaluate AI models’ perception and reproduction of abstract visual language.

By Haowei Zhang, Yuanpei Zhao, Ji-Zhe Zhou, Mao Li
arXiv Computation and Language
Aug 31

The Telephone Game: Evaluating Semantic Drift in Unified Models

The paper introduces the Semantic Drift Protocol (SDP), a Telephone Game-inspired method to evaluate how well unified models preserve meaning when alternating between image-to-text (I2T) and text-to-image (T2I) generation over multiple generations. It defines Mean Cumulative Drift (MCD) and Multi-Generation GenEval (MGG) as metrics for semantic retention, and presents a new benchmark of 400 image‑text pairs from NoCaps and DOCCI to stress-test models beyond COCO. Applying SDP to seven models shows that models with strong single‑pass scores can still suffer severe semantic drift, revealing catastrophic failure modes that isolated benchmarks miss.

By Sabbir Mollah, Rohit Gupta, Sirnam Swetha, Qingyang Liu, Ahnaf Munir, Mubarak Shah
arXiv Machine Learning
Aug 31

Accelerating LLM Inference via Vector Index Based Output Embeddings

The paper proposes replacing dense output projection in large language models with an HNSW-based vector index to perform maximum inner product search over token embeddings. This approach reduces memory bandwidth usage by retrieving only a small set of high-scoring tokens and can be integrated into existing decoding pipelines via sparse logits scattering. Experiments on Gemma 3, Llama 3.2, and Qwen 3 show up to 82% speed‑up in batch‑size‑one decoding while maintaining generation quality.

By Martin Loretz, Sepp Hochreiter
arXiv Computation and Language
Aug 31

What Makes Agent Memory Useful for Reliable Unanswerable Question Handling?

The paper investigates how agent memory contributes to reliable handling of unanswerable questions (UAQs) within a unified Retrieval-Augmented Generation (RAG) framework. Four memory methods were evaluated across three UAQ datasets and two base models, revealing that memory can improve UAQ performance in selective settings but the gains are fragile under dataset shift. Procedural and rule-based memories, especially when combined with complementary behavioral signals, provide the most reliable support, indicating that effective UAQ memory relies more on transferable behavioral guidance than on sheer volume of stored experience.

By Chuanyuan Tan, Junjie Yu, Yuxin Wang, Yining Zheng, Xipeng Qiu, Wenliang Chen
arXiv Computation and Language
Aug 31

FinExam-10K: When Retrieval Helps Financial Reasoning?

FinExam-10K is a new English benchmark for financial reasoning, comprising 10,198 expert‑reannotated questions covering CFA Levels I‑III and FRM Parts I‑II. The dataset is split into a 5,110‑question release and a 5,088‑question held‑out set for a quarterly leaderboard, with separate Full‑Coverage and Context‑Complete Reasoning tracks. Across 17 models, the best overall accuracy is 85.29 %, but performance drops on harder subsets, and retrieval‑augmented methods like Function‑Graph‑RAG provide modest gains when gated appropriately.

By Yan Lin, Jingyu Sun, Zhongliang Guo, Qing Li, Zhuohan Xie, Yuxia Wang
arXiv Machine Learning
Aug 31

EXPOSE: Explainable and Domain-Robust Embeddings from Pathology Vision Foundation Models using Sparse Autoencoders

EXPOSE is a framework that applies Sparse Autoencoders to Vision Foundation Model embeddings in computational pathology, aiming to separate biological signals from domain‑specific noise. By training a sparse representation of VFM features and using a linear classifier to flag domain‑specific latent dimensions, the method masks these components before downstream relapse prediction, avoiding the need to retrain the backbone model. Experiments on a large prostate cancer dataset demonstrate that removing domain‑specific features improves cross‑domain performance and raises the Domain Robustness Index (DoRI).

By Anja Witte, Maximilian Lennartz, Jan Baumbach, Guido Sauter, Stefan Bonn, Patrick Fuhlert, Marina Zimmermann
arXiv Machine Learning
Aug 31

Quantum SEDONet: Spectrally-Embedded Quantum Deep Operator Networks for Partial Differential Equations

Quantum SEDONet is a quantum-enhanced deep operator network that embeds spectral features—Fourier for periodic coordinates and Chebyshev for bounded, non‑periodic coordinates—directly into the trunk network. This coordinate‑wise spectral embedding is achieved without adding qubits or circuit depth under unary amplitude encoding, and it reduces mean relative L2 error by up to 54.1% across four PDE benchmarks compared to the baseline Quantum DeepONet. The method demonstrates that quantum and classical inference paths agree to within 10⁻⁸, and it allows simultaneous use of both spectral bases within a single problem, as shown in a mixed‑boundary Poisson channel example.

By Muhammad Abid, Arth Sojitra, Bipin Tiwari, Omer San