Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,124 stories · RSS feed

arXiv Computer Vision
6d ago

VTrajEmbed: Aligning Text with Video Object Trajectories via Spatiotemporal Embeddings

VTrajEmbed is an embedding model that aligns text with video object trajectories by decomposing videos into individual trajectories and embedding each trajectory separately. It introduces a Slow‑Fast Trajectory Encoding dual‑branch architecture that captures high‑frequency motion and scene context, and a Mask‑Guided Contrastive Learning objective that prevents false negatives during training. The model is evaluated on a new trajectory‑level benchmark, Text2Traj‑Bench, and outperforms existing large‑language‑model embeddings by significant margins on both text‑to‑trajectory and trajectory‑to‑text retrieval tasks.

By Ruizhi Chen, Zining Fan, Yudi Guo, Jiang Liu, Zhiyu Lin, Zhongjiang He, Shouqian Shi, Hao Sun
arXiv AI
6d ago

Characterising and Analysing Retrieval-Augmented Generation Techniques

The paper introduces a conceptual framework to classify and evaluate Retrieval-Augmented Generation (RAG) techniques, addressing the challenge of selecting suitable methods for specific tasks. It identifies main dimensions and sub-dimensions based on a qualitative analysis of existing literature. The framework can act as a pre-empirical filter, helping researchers and practitioners choose the most fitting RAG candidates before costly benchmarking.

By Emmanuel Igbekele, Olawande Daramola
arXiv AI
6d ago

AuditKIT: A Unified Evaluation Spine for Benchmarking, Judging, and Comparing Large Language Models and Agentic Systems

AuditKIT is a source‑available evaluation library that unifies benchmark accuracy, judge‑based assessment, runtime performance, tool‑use behavior, and retrieval quality into a single result representation. It offers direction‑aware metric contracts, configuration fingerprints, and extensible metrics and backends, while providing turn‑aware tool‑call matching and distinguishing verified task outcomes from trajectory‑level diagnostics. Seven case studies demonstrate how answer extraction, runtime behavior, and metric selection can alter evaluation outcomes across model comparison, serving runtimes, retrieval, tool calling, and interoperability.

By Shreeyans Arora, Utsav Avaiya, Zera Lyngkhoi, Hem Gosalia, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav Seth
arXiv AI
6d ago

Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning

The paper introduces JEPA‑Prune, a post‑hoc neuron‑pruning method that mitigates private data leakage in fine‑tuned large language models. It trains a public model on PII‑free surrogate views using a joint‑embedding predictive architecture, then uses cross‑model representation probes, integrated gradients, and a specificity score to identify and prune neurons that respond to personally identifiable information. Experiments show that pruning only 1.6% of neurons can dramatically reduce PII inference accuracy and recall while preserving model utility, outperforming existing privacy‑neuron editing techniques.

By Mohamed Shaaban, Mohamed Elmahallawy
arXiv AI
6d ago

Accurate and Efficient Multi-Channel Time Series Forecasting via Sparse Attention Mechanism

The paper introduces Li‑Net, a new architecture for multi‑channel time series forecasting that captures both linear and non‑linear dependencies among channels. Li‑Net compresses representations across sequence and channel dimensions, processes them with a configurable non‑linear module, and reconstructs forecasts while integrating a sparse Top‑K Softmax attention mechanism within a multi‑scale projection framework. Experiments on real‑world benchmarks show that Li‑Net matches state‑of‑the‑art accuracy while using less memory and achieving faster inference, with ablation studies confirming the contribution of each component.

By Lei Gao, Hengda Bao, Jingfei Fang, Guangzheng Wu, Weihua Zhou, Yun Zhou
arXiv AI
6d ago

CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising

The paper introduces CDAE, a lightweight Contrastive Denoising Autoencoder that refines pre-trained BERT embeddings by jointly optimizing contrastive and reconstruction objectives. It aims to produce perturbation‑invariant sentence representations, addressing sensitivity to textual perturbations such as synonym substitution, masking, and word dropout. Experiments across various perturbation strategies and strengths show that CDAE consistently maintains higher embedding similarity than original BERT and SimCSE, with the advantage growing as perturbation strength increases.

By Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand, Majid Ramezani
arXiv AI
6d ago

STRIDE: Reasoning-Aware Training for Time Series Forecasting

STRIDE (Strategic Time-series Reasoning Injected via Distilled Embeddings) is a framework that embeds LLM reasoning directly into the continuous embedding space of Time Series Foundation Models (TSFMs). By distilling reasoning traces into a lightweight LLM and projecting its pooled hidden states as a continuous cross‑modal prior, STRIDE integrates semantic reasoning without fragmenting numerical values or inflating sequence lengths. The method is jointly optimized with cross‑entropy and quantile losses, can be attached to various TSFM architectures, and achieves top performance on the GIFT‑Eval leaderboard and strong results on TFRBench, with ablations showing that the gains come from strategic reasoning integration.

By Md Atik Ahamed, Mihir Parmar, Palash Goyal, Chun-Liang Li, Qiang Cheng, Tomas Pfister, Jinsung Yoon
arXiv Machine Learning
6d ago

Noise-shaped Sampling and Annealing for Scalable On-the-edge Training of Binary and Ternary AI Models

arXiv:2610. 22621v1 Announce Type: new Abstract: The training loss of binary and ternary neural networks, expanded as a multilinear polynomial over the discrete weights, is structurally equivalent to the energy of a hierarchical high-order Ising model.

By Zihao Chen, Zhili Xiao, Rui-Jie Zhu, Yipeng Wang, Jason Eshraghian, Shantanu Chakrabartty
arXiv Machine Learning
6d ago

Sentence-Level Topic Modeling with Lexical and Contextual Evidence

SentenceETM is a neural topic model that assigns a topic to each sentence by jointly using lexical and contextual evidence. It represents each topic with a vocabulary distribution via word embeddings and a Gaussian over sentence embeddings in a topic‑specific low‑dimensional subspace. The model exactly marginalizes the sentence topic, combining word and embedding evidence, and uses amortized variational inference for document proportions while re‑estimating Gaussian parameters in closed form.

By Federica Granese, Serena Villata, Charles Bouveyron
arXiv Machine Learning
6d ago

Interpreting SSL Representations for Spoof Detection: a WavLM Study

The study investigates which layers of the self‑supervised speech model WavLM carry the most spoof‑relevant information for spoof detection. Using the ASVspoof 5 database, the authors perform per‑layer linear probing, canonical correlation analysis (CCA) with acoustic features, and a grid search over layer subsets, finding that five specific layers outperform pooling all thirteen. CCA shows mid‑layers align with most acoustic features, while voice‑quality metrics correlate more with later layers, and visualizations suggest deeper layers encode general synthesis artifacts rather than attack‑specific signatures.

By Mohamed Mallat, Michele Panariello, Massimiliano Todisco, Nicholas Evans, Anthony Larcher
arXiv Machine Learning
6d ago

GARG-AML against Smurfing: A Scalable and Interpretable Graph-Based Framework for Anti-Money Laundering

The paper introduces GARG-AML, a graph-based framework designed to detect smurfing—a money‑laundering technique that splits large transactions into smaller ones—to improve anti‑money laundering efforts. It quantifies smurfing risk with a single interpretable metric derived from the second‑order transaction network, offering a balance between computational efficiency and transparency. The authors demonstrate GARG‑AML’s scalability on large random networks and its effectiveness on two open‑source money‑laundering datasets, suggesting it can enhance smurfing detection and support investigative decision‑making.

By Bruno Deprez, Bart Baesens, Tim Verdonck, Wouter Verbeke
arXiv Machine Learning
6d ago

DisCo Algebra: Why Disintegration and Combination Are Not Inverses

arXiv:2610. 15006v2 Announce Type: replace-cross Abstract: We present DisCo algebra---a compositional framework for joint probability built from a symmetric cooperad (disintegration) and a non-unital partial symmetric operad (combination restricted to the image of disintegration) on the bare family $\mathcal{P}_n=\mathrm{Prob}([0,1]^n)$.

By Xuexing Lu
arXiv Machine Learning
6d ago

Dependency Reasoning for Aspect Sentiment Quadruple Prediction

The paper introduces DRIFT, a method that linearizes dependency parse trees and injects them into large language model prompts to improve Aspect Sentiment Quadruple Prediction (ASQP). By providing explicit relational context, DRIFT enables the model to reason over aspect‑opinion dependencies rather than relying on token proximity. Experiments on SemEval Rest15, Rest16, and ACOS show that DRIFT outperforms other generative ASQP methods, achieving 54.47% and 64.10% F1 on Rest15 and Rest16, and demonstrating consistent gains across short- and medium‑distance aspect‑opinion configurations, while also revealing that exact‑match metrics underestimate semantically correct predictions.

By Shima Esfandyari, Seywan Soleymanbaigi, Fardin Akhlaghian Tab, Fatemeh Daneshfar, Mourad Oussalah
arXiv Computation and Language
6d ago

When Does Uncertainty Measure RAG Hallucination Risk?

The paper investigates whether uncertainty estimates can gauge hallucination risk in retrieval‑augmented generation (RAG). It finds that uncertainty is informative only when retrieved evidence constrains the answer: with good evidence, unsupported claims are less likely, whereas poor retrieval coverage lets the model rely on internal knowledge and assign high probability to unsupported claims. Experiments on HotpotQA and SQuAD across four instruction‑tuned models show that perplexity‑based uncertainty achieves AUROC scores of 0.81–0.91 for detecting targeted unsupported edits, while length‑only classifiers perform near chance. The study also demonstrates that adding a second passage can recover detection performance and that combining uncertainty with a semantic‑drift score further improves calibration.

By Rajeel Ahmad Ansari, Tautik Agrahari
arXiv Computation and Language
6d ago

Inductive Learning of Concept Embeddings from Cross-Linguistic Colexification Patterns with Textual Attributes

The paper introduces an inductive method for learning concept embeddings from cross‑linguistic colexification networks combined with short textual descriptions. Unlike previous transductive approaches, this framework can embed unseen concepts using only a brief definition. The resulting embeddings correlate with cross‑linguistic similarity measures and generalize well to new concepts.

By Arne Rubehn
arXiv Computation and Language
6d ago

ProPerAgent: Toward Memory-Driven Proactive Assistance

The paper introduces ProPerMem, a benchmark for memory-driven proactive assistance that pairs egocentric video with long, distractor-rich personal memories and human-verified trigger windows. Baseline models perform poorly, achieving at most 15% accuracy, while an oracle-driven GPT-5.3-Chat model reaches 96% when given evidence inside the trigger window. Building on these insights, the authors develop ProPerAgent, a plug‑and‑play system that retrieves relevant memory from a graph store and uses lightweight reinforcement learning to improve proactive intervention accuracy to 42.4% without training.

By Shenglai Zeng, Xiao Yang, Tiffany Ma, Xinyuan Zhang, Nanshu Wang, Deren Lei, Kai Sun, Xilun Chen, Tammy Stark, Xianhui Zhu, Yuetai Li, Rulin Shao, Tianxin Wei, Rakesh Wanga, Anuj Kumar, Jiliang Tang, Wen-tau Yih, Xin Luna Dong
arXiv Computation and Language
6d ago

ConvexLLM: Scaling Convex-Hull Speech Interfaces to General-Purpose Audio Intelligence

ConvexLLM introduces a C‑Gate interface that represents audio as continuous convex combinations of frozen text embeddings, aiming to preserve pretrained text capabilities while incorporating acoustic information. The model achieves a VoiceBench aggregate score of 75.36 and an MMAU Test‑mini accuracy of 62.50%, outperforming MLP and Q‑Former interfaces on spoken knowledge and reasoning tasks. Compared to a 2B‑parameter decoder, C‑Gate delivers higher spoken‑language performance with comparable text‑task results and task‑dependent tradeoffs in transcription and audio understanding.

By Ming-Hao Hsu, Shujie Liu, Yuxuan Hu, Yuxiang Wang, Yingda Shen, Jinyu Li, Zhizheng Wu
arXiv Machine Learning
6d ago

From User Recognition to Activity Counting: An Identity-Agnostic Approach to Multi-User WiFi Sensing

The paper proposes an identity‑agnostic approach to multi‑user WiFi sensing that reframes activity recognition as counting how many users perform each activity, rather than assigning actions to specific individuals. Using WiMANS data, the identity‑agnostic counting model outperforms identity‑dependent predictions on integer‑count mean absolute error across all tested environments, and remains robust when users are unseen. Embedding analysis shows that identity‑agnostic representations cluster by activity composition instead of user identity, supporting the claim that counting is a more accurate and robust scene‑level estimation method.

By Kemal Bayik, Olayinka Ajayi, Daniel Roggen, Philip Birch
arXiv Computer Vision
6d ago

Stuck on Shuffle: On Attribute-Object Binding in Vision-Language Models

The paper introduces SABER, a benchmark that tests attribute–object binding in vision‑language models across all four modality pairings and scenes with up to ten objects. Experiments show significant binding failures in both CLIP‑style dual encoders and recent MLLM‑based embedders. The authors propose a binding‑focused fine‑tuning strategy using synthetic attribute‑binding text, which improves SABER performance and also boosts results on negation and instance‑retrieval benchmarks.

By Netanel Y. Tamir, Michael Green, Issar Tzachor, Or Shimshi, Dvir Samuel, Or Patashnik, Rami Ben-Ari
arXiv Computer Vision
6d ago

Kingfisher: Fast Personalized Visual Autoregressive Generation via Scale-Aware Prompt-Conditioned Scale Decomposition

Kingfisher is a fast, model‑tuning‑free framework for personalized image generation using visual autoregressive models. It decomposes early autoregressive scale codes into content and style components conditioned on prompts, then recombines these components from two real images into a compact semantic prefix. The frozen generator refines the remaining scales, achieving strong content preservation and superior style and text alignment while cutting pair‑specific preparation time by 54× compared with diffusion methods and 7.5× compared with other VAR approaches.

By Dinh-Khoi Vo, Akihiro Sugimoto, Minh-Triet Tran, Trung-Nghia Le