Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,383 stories · RSS feed

arXiv Machine Learning
Aug 31

EXPOSE: Explainable and Domain-Robust Embeddings from Pathology Vision Foundation Models using Sparse Autoencoders

EXPOSE is a framework that applies Sparse Autoencoders to Vision Foundation Model embeddings in computational pathology, aiming to separate biological signals from domain‑specific noise. By training a sparse representation of VFM features and using a linear classifier to flag domain‑specific latent dimensions, the method masks these components before downstream relapse prediction, avoiding the need to retrain the backbone model. Experiments on a large prostate cancer dataset demonstrate that removing domain‑specific features improves cross‑domain performance and raises the Domain Robustness Index (DoRI).

By Anja Witte, Maximilian Lennartz, Jan Baumbach, Guido Sauter, Stefan Bonn, Patrick Fuhlert, Marina Zimmermann
arXiv Computation and Language
Aug 31

The Telephone Game: Evaluating Semantic Drift in Unified Models

The paper introduces the Semantic Drift Protocol (SDP), a Telephone Game-inspired method to evaluate how well unified models preserve meaning when alternating between image-to-text (I2T) and text-to-image (T2I) generation over multiple generations. It defines Mean Cumulative Drift (MCD) and Multi-Generation GenEval (MGG) as metrics for semantic retention, and presents a new benchmark of 400 image‑text pairs from NoCaps and DOCCI to stress-test models beyond COCO. Applying SDP to seven models shows that models with strong single‑pass scores can still suffer severe semantic drift, revealing catastrophic failure modes that isolated benchmarks miss.

By Sabbir Mollah, Rohit Gupta, Sirnam Swetha, Qingyang Liu, Ahnaf Munir, Mubarak Shah
arXiv Machine Learning
Aug 31

Leveraging a Foundation Model for the EEG-Based Diagnosis of Alzheimer's Disease

The paper presents a diagnostic framework for Alzheimer’s disease that uses the Large Brain Model (LaBraM), a foundation model pretrained on over 2,500 hours of EEG data, to generate high‑dimensional latent embeddings. These embeddings are fed into a non‑linear Random Forest classifier, achieving an ROC‑AUC of 89.36% ± 3.49%, PR AUC of 81.45% ± 4.43%, and Balanced Accuracy of 82.44% ± 4.34% in a subject‑independent 5‑fold cross‑validation setting, using only 8‑second EEG segments. Post‑hoc occlusion and neurophysiological alignment analyses confirm that the model captures clinically validated biomarkers such as occipital‑frontal Alpha and Theta rhythm degradation and correlates with cognitive performance and clinical severity.

By Maggie Lin, Chung-Lin Hou, Tzyy-Ping Jung
arXiv Computer Vision
Aug 31

Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision

The study investigates whether transferring relational structure from human mental representations to deep neural networks (DNNs) can improve fine‑grained alignment between the two. Using unsupervised Gromov‑Wasserstein optimal transport, the authors show that fine‑tuning pre‑trained DNNs with Relational Knowledge Distillation (RKD) brings the networks close enough to human representations to align at the individual‑object level on a test set of concepts not seen during training. The improvement is driven mainly by a more human‑like global structure of category distances, while local nearest‑neighbor overlap remains largely unchanged.

By Yuria Shimizu, Soh Takahashi, Takato Horii, Masafumi Oizumi
arXiv Computer Vision
Aug 31

WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild

WilLaGS introduces a unified framework that enhances 3D Gaussian Splatting for in-the-wild scenes by learning a continuous global appearance manifold with a β‑VAE and generating dynamic Tri‑Plane features for spatially‑varying local illumination. It also incorporates a self‑supervised perceptual masking mechanism using a Teacher‑Student EMA architecture to suppress transient artifacts and identify inconsistent regions. Experiments on multiple datasets show that WilLaGS achieves state‑of‑the‑art reconstruction quality and novel view synthesis while preserving real‑time rendering efficiency.

By Yuhao Bai, Qianqiu Tan, Lilong Chen, Huanhuan Lv, Lijun Chen
arXiv Computation and Language
Aug 31

Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations

The paper investigates the intrinsic dimension (ID) of large language model (LLM) representations as an indicator of linguistic complexity. By comparing ID across model layers for coordination vs. subordination, right‑branching vs. center‑embedding, and unambiguous vs. ambiguous attachment, the authors find consistent ID differences that align with established complexity contrasts. Experiments across six LLMs, including representational similarity and layer pruning analyses, confirm that more complex phenomena produce higher ID profiles, with peaks occurring at different layers for each contrast.

By Marco Baroni, Emily Cheng, Iria de-Dios-Flores, Francesca Franzon
arXiv Computation and Language
Aug 31

Nested Byte-Level Vocabularies Are Cheap to Deploy and Expensive to Share: A Pre-Registered Negative Result

The paper demonstrates that a byte‑level BPE tokenizer can be sliced to create multiple vocabulary sizes from a single trained model, preserving exact logits while reducing deployed weights by 66%. Experiments on 30 models show that while sliced models match the full model numerically, they underperform fixed‑cap specialists by a few percentage points in bits‑per‑byte. Multi‑cap training improves robustness to typographical noise, suggesting benefits from training across multiple granularities rather than from control tokens alone.

By Christos Koutsiaris
Towards Data Science
Aug 30

Noisy Text in RAG: Typos, OCR, and the Gap Classical Spell-Check Leaves

The article discusses how noisy text—stemming from user typos, rapid transcription errors, and OCR character mistakes—poses challenges for Retrieval-Augmented Generation (RAG) systems. It explains that traditional spell-checking only addresses one type of error, while embeddings are needed to handle the remaining noise. The piece highlights the need for more robust solutions in enterprise document intelligence.

By Kezhan Shi
Towards Data Science
Aug 29

RAG Is Not the Whole Toolkit: The NLP Techniques Real Problems Still Need

The article argues that Retrieval-Augmented Generation (RAG) is only one tool in NLP, and many real-world problems—such as request classification, free‑text matching, table reading, and OCR noise cleaning—are better served by simpler, cheaper techniques. It emphasizes the importance of selecting the appropriate method for each task and highlights the engineering challenge of knowing which technique to apply.

By Kezhan Shi
arXiv Machine Learning
Aug 28

hoBIT: A Profile-Aware Retrieval-Augmented Chatbot for University Academic Advising

The paper introduces proFILL, a method that transforms the existing rule-based advising chatbot hoBIT into a profile-aware retrieval-augmented generation system. Instead of needing a full student profile at the start, proFILL incrementally gathers only the attributes required for each query, using both the query intent and initially retrieved evidence to guide retrieval from a profile-aware index. Experiments and a human preference study demonstrate that proFILL outperforms various RAG baselines, is favored by users, and remains effective when deployed with open-weight models for cost-efficient on‑premise use.

By Yoonseo Kim, Seongmin Lee, Joongheon Kim, SeongKu Kang
arXiv AI
Aug 28

The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection

The paper introduces the Latent Diagnostic Taxonomy, a framework that builds a dimensionality‑optimized classifier and a diagnostic tool to assess the trustworthiness of its confident predictions. It identifies a small set of influential prompts (latent support vectors) that reveal tokens which can change the classifier’s output, and uses these tokens to create a taxonomy that classifies prompts into safe, heuristic bias, heuristic override, or insufficient context categories. Applied to a prompt‑injection detection model, the framework shows that about 77% of confident decisions are fragile to a single token, distinguishing between calibration failures and exploitable shortcuts, and offers remediation strategies for each taxonomy zone.

By Jaturong Kongmanee, Smile Thanapattheerakul
arXiv AI
Aug 28

Multivariate Diffusion Transformer with Decoupled Attention for High-Fidelity Mask-Text Collaborative Facial Generation

The paper introduces MDiTFace, a diffusion transformer designed for high‑fidelity mask‑text collaborative facial generation. It unifies tokenization of semantic masks and text, enabling synchronous multimodal feature interaction via stacked multivariate transformer blocks. A novel decoupled attention mechanism separates dynamic and static computations, allowing caching of static features and reducing mask‑condition overhead by over 94% while preserving performance, leading to superior facial fidelity and conditional consistency compared to existing methods.

By Yushe Cao, Dianxi Shi, Xing Fu, Xuechao Zou, Haikuo Peng, Xueqi Li, Chun Yu, Junliang Xing
arXiv Computation and Language
Aug 28

TabuLM: Morphology-Aware Tabular Pre-training for Low-Resource Languages

TabuLM is a new language model pre‑trained on Kinyarwanda tabular data, extending KinyaBERT‑large with row, column, and cell‑type embeddings and a table‑structure attention bias. It introduces two pre‑training objectives—Masked Cell Recovery and Column Type Prediction—and is trained on 172 Rwandan government tables. On the TabQA‑kin benchmark, TabuLM achieves 62.0% exact match, outperforming KinyaBERT‑large and multilingual baselines by significant margins.

By Ireddi Rakshitha, Devavarapu Yashwanth, Ntakirutimana Pierre
arXiv AI
Aug 28

MIMO: Multilingual Information Retrieval via Monolingual Objectives

MIMO: Multilingual Information Retrieval via Monolingual Objectives proposes a two‑stage framework that first aligns a student model to a stable English semantic space using knowledge distillation, then jointly optimizes distillation and cross‑lingual contrastive learning to improve retrieval discrimination while preserving alignment. The approach addresses language clustering and the trade‑off between cross‑lingual alignment and embedding uniformity, outperforming existing cross‑lingual training baselines on both multilingual and multi‑monolingual benchmarks. MIMO also remains competitive with larger off‑the‑shelf models and its alignment‑uniformity analysis clarifies the distinct roles of the two loss components. whyItMatters":"The study demonstrates a practical method to enhance multilingual information retrieval performance by balancing alignment and uniformity, which is crucial for real‑world search environments where queries and documents span multiple languages."

By Youngjoon Jang, Seongtae Hong, Heuiseok Lim
arXiv Computer Vision
Aug 28

Sidecar: Training-Free Semantic Reuse for Character-Consistent Free-form Visual Storytelling

Sidecar is a plug‑and‑play semantic augmentation module that preserves entity‑level information from a character’s initial description and injects missing semantics into later prompt embeddings, enabling training‑free character consistency in free‑form visual storytelling. It works with existing diffusion models without architectural changes and incurs negligible computational overhead. Experiments on FreeStoryBench demonstrate consistent improvements in prompt‑image alignment and character consistency across SDXL and FLUX baselines.

By Sibo Dong, Sarah Adel Bargal