Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,383 stories · RSS feed

arXiv AI
Aug 26

Predicting Radiologist Expertise from 3D Gaze Patterns During CT Interpretation

The study introduces a gaze-informed transformer framework that classifies radiologist expertise during thoracic CT interpretation by integrating eye‑tracking data into volumetric feature learning. Using a DINOv2 backbone, the model incorporates a learnable log‑space bias in self‑attention and gaze‑weighted pooling of patch embeddings. Trained on 182 CT reading sessions from five radiologists, it achieved an ROC‑AUC of 0.91 and an F1 score of 0.86, outperforming adapted baseline methods.

By Leila Khaertdinova, Anna Anikina, Claudia Mello-Thoms, Bulat Ibragimov
arXiv AI
Aug 26

Generative AI for Validating Physics Laws

arXiv:2503.17894v3 Announce Type: replace-cross Abstract: We propose generative learner for estimating heterogeneous treatment effects and characterizing the full distribution of causal effects. The...

By Maria Nareklishvili, Nicholas Polson, Vadim Sokolov
arXiv Computer Vision
Aug 26

What Does Prompt Learning Change? -A Natural-Language Concept Analysis of Vision-Language Models

Prompt learning modifies vision‑language models by optimizing continuous prompt vectors, yet the resulting prompts are hard to interpret in natural language. PromptSpLiCE is a post‑hoc method that rewrites each class‑conditioned text embedding as a sparse mix of concepts from a fixed dictionary, enabling a direct comparison of concept profiles before and after prompt learning. Across 11 image‑classification datasets, the method shows that only about 1.6 of the initial top‑10 concepts remain after learning, and that larger profile changes correlate with higher accuracy gains, while a derived gradient expression offers geometric insight into loss sensitivity.

By Ryo Kamiya, Hiroshi Kera, Kazuhiko Kawamoto
arXiv AI
Aug 26

Relative Time Intervals Representation for Word-level Timestamping with Masked Training

The paper introduces a method for improving fine-grained, temporally aligned outputs in Speech Large Language Models (SpeechLLMs) by replacing absolute timestamps with relative timestamps, which reduces vocabulary size and enhances generalization. It proposes a hybrid fine‑tuning strategy that fully fine‑tunes the timestamp‑augmented embedding layer and language model head while applying LoRA to decoder layers, and introduces a masked timestamp training objective to prevent over‑reliance on ground‑truth timestamps. Experiments show significant gains in timestamp prediction accuracy without compromising transcription quality.

By Quanwei Tang, Zhiyu Tang, Xu Li, Dong Zhang, Shoushan, Guodong Zhou
arXiv Machine Learning
Aug 26

A Multimodal Foundation Model for Longitudinal Patient Representation and Scalable Insight Generation in Oncology

arXiv:2608.24688v1 Announce Type: new Abstract: Precision oncology necessitates a longitudinal model of patient state that captures cancer evolution and treatment over time, integrating multimodal ob...

By Eugene Vorontsov, Yi Kan Wang, Alican Bozkurt, Adam Casson, Ludmila Tydlitatova, Michal Zelechowski, Ezra E. W. Cohen, Jyoti D. Patel, Max Banaszak, Caitlin McWilliams, Shane Colley, Kate Sasser, Ryan Fukushima, Eric Lefkofsky, Razik Yousfi, Siqi Liu
arXiv AI
Aug 26

Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation

Hybrid Semantic Tool Discovery for Enterprise MCP Gateway presents SCOUT, a system that addresses two major challenges in large language model (LLM) agent tool usage: a context‑engineering bottleneck and a tool discoverability barrier. SCOUT reframes tool exposure as a context‑selection problem, injecting only relevant tools into the model’s context window and providing two MCP meta‑tools—tool_search and execute_tool—to perform hybrid retrieval via BM25 and dense vector search. In production at PayPal, SCOUT cuts MCP tool‑token consumption by 99%, dramatically reducing per‑query inference cost while remaining model‑agnostic and requiring no client‑side changes.

By Olympia Saha, Amy Wang, Srinivasan Manoharan
arXiv AI
Aug 26

Mind the Student: Behavioral and Contextual Cues for Automated Engagement Prediction in Online Learning

The paper tackles the challenge of predicting student engagement from online tutoring videos, noting that engagement is a complex, multidimensional construct influenced by behavioral, emotional, and cognitive states. By analyzing the CASED dataset, the authors highlight the difficulty posed by high inter‑person variability and subjective annotations. They propose a multimodal framework that fuses implicit spatiotemporal features from pretrained video, audio, and image encoders with structured behavioral cues such as head pose, gaze, facial action units, emotion, and wavelet‑based audio features, integrating them via a Perceiver IO bottleneck and modeling participant personalities with variational posteriors. The system employs evidential regression and spectral‑normalized Gaussian process classification heads to provide uncertainty‑aware predictions, achieving competitive performance on the CASED challenge test set while offering well‑calibrated uncertainty metrics.

By Alperen Kantarci, Visvanathan Ramesh, Gemma Roig
arXiv Machine Learning
Aug 26

NAIMA: Semantics Aware RGB Guided Depth Super-Resolution

The paper introduces NAIMA, a guided depth super‑resolution framework that leverages global contextual semantic priors from pretrained vision transformer token embeddings. Its Guided Token Attention (GTA) module uses depth encodings as queries to attend over semantic tokens, with a zero‑initialized gate controlling the influence of semantic evidence. NAIMA achieves competitive in‑distribution performance while delivering superior cross‑dataset generalization without relying on decoded priors or auxiliary objectives.

By Tayyab Nasir, Daochang Liu, Ajmal Mian
arXiv AI
Aug 26

Evidence Blindness in Direct Corpus Interaction: Persistent Navigation with AtlasNav

The paper introduces AtlasNav, a persistent multi‑view corpus‑navigation framework that organizes a corpus into a Corpus Atlas, enabling large‑language‑model agents to navigate efficiently under finite interaction budgets. AtlasNav reduces online inference cost by 30.21% and achieves 92.05% strict accuracy on BrowseComp‑Plus, while earlier and more rapidly realizing required evidence compared to dynamic‑workspace methods. The approach also transfers well to other corpora such as PhantomWiki and heterogeneous enterprise knowledge bases, demonstrating that effective agentic search relies on both accessible evidence and a reusable corpus representation.

By Hongyu Guo, Zhiyu Zheng, Zhao Cao
arXiv AI
Aug 26

Retrieval-augmented generation vs. deterministic tax computation in multi-agent financial advisory: A 2x2 factorial experiment

The study compares a custom capital gains calculation engine with a retrieval‑augmented generation (RAG) vector store of market advisory reports in a multi‑agent financial advisory system. A 2x2 factorial experiment showed that the tax‑optimization engine significantly reduced tax savings, while the RAG component had no significant effect. The RAG‑only condition yielded the highest tax savings, suggesting that pretrained language model knowledge may suffice for tax‑loss harvesting without specialized tooling.

By Aryan Brar, Justin Du, Avery Lor, Kylie Seto, Eric Taylor
arXiv AI
Aug 26

Strictly Causal Streaming Video Anomaly Detection with a Theoretically-Grounded State-Space Core

The paper presents a strictly causal streaming video anomaly detector that updates a fixed‑size state in constant time per frame, eliminating the need for clip buffering or lookahead. Its core is a diagonal linear state‑space recurrence with a decay gate, trained via self‑supervised next‑embedding prediction on a frozen visual backbone. The authors derive a closed‑form link between the recurrence’s decay spectrum and detection delay, validate on UCSD Ped2 and CUHK Avenue, and report real‑time latency on Apple M3 Pro hardware (≈0.75 ms per frame).

By Yogesh Kumar