Towards Data Science

Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions

The article reports on an experiment where an AI agent was allowed to search for evidence whenever it chose. The agent discovered some papers that a standard search missed, but also lost some that the plain search had found, and overall the gains were insufficient to be trusted. The post examines whether agentic Retrieval-Augmented Generation (RAG) truly retrieves better evidence across 298 questions.

Towards Data Science
Oct 1

Can an Apartment Search Agent Call the Model Fewer Times and Still Find Good Matches?

The article examines whether an apartment search agent can reduce the number of times it calls a predictive model while still identifying suitable matches. The author conducted 2,500 listing checks using Weights & Biases Weave, systematically eliminating unnecessary model computations, and evaluated each iteration against consistent reference answers.

By Abdullahi Dattijo
Towards Data Science
Sep 2

A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence

The article discusses the importance of a Retrieval-Augmented Generation (RAG) system providing clear evidence when it states that information is not present in a document. It outlines four distinct types of evidence that should accompany such a claim to avoid presenting a confident but incorrect answer or an unsupported “no answer.” The piece emphasizes that each evidence type serves as a safeguard against misinformation in enterprise document intelligence.

By Kezhan Shi
arXiv AI
Oct 16

Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence

The paper investigates why large language models (LLMs) sometimes give outdated answers even when recent evidence is available through web search. By creating temporal facts that changed after the models’ training cutoffs, the authors evaluate a proprietary model with deployed search and six open-weight models with controlled retrieval. They find that models often rely on search snippets without opening source pages, leading to more outdated answers and unfaithful citations, and that even when retrieved evidence influences internal predictions, the final answer may still favor outdated information.

By Owen Walters, Lihu Chen, Francesca Toni
Towards Data Science
Aug 29

RAG Is Not the Whole Toolkit: The NLP Techniques Real Problems Still Need

The article argues that Retrieval-Augmented Generation (RAG) is only one tool in NLP, and many real-world problems—such as request classification, free‑text matching, table reading, and OCR noise cleaning—are better served by simpler, cheaper techniques. It emphasizes the importance of selecting the appropriate method for each task and highlights the engineering challenge of knowing which technique to apply.

By Kezhan Shi
Towards Data Science
Aug 27

Agentic AI Is Rewriting The Analytics Stack But There's One Skill It Still Can't Touch

The article discusses how agentic AI is reshaping the analytics stack by taking over more execution tasks. It raises the question of which responsibilities should remain with human analysts versus AI agents and explores the importance of this distinction. The piece highlights the evolving role of AI in analytics and the need to define clear boundaries between human and machine work.

By Rashi Desai