Towards Data Science

Why a Tiny GPT Can Know Two Facts and Still Not Connect Them

A tiny GPT model was trained to memorize every fact presented to it, yet it failed to answer any question that required combining two of those facts. The author tested a one-line fix suggested in a recent paper, which only partially resolved the issue. The post highlights the limitations of small language models in integrating multiple pieces of information.

Towards Data Science
Sep 2

A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence

The article discusses the importance of a Retrieval-Augmented Generation (RAG) system providing clear evidence when it states that information is not present in a document. It outlines four distinct types of evidence that should accompany such a claim to avoid presenting a confident but incorrect answer or an unsupported “no answer.” The piece emphasizes that each evidence type serves as a safeguard against misinformation in enterprise document intelligence.

By Kezhan Shi
Towards Data Science
Oct 6

A Google Team Measured Half of My Argument, and Left the Other Half Open

The article titled "A Google Team Measured Half of My Argument, and Left the Other Half Open" is part of a series on spec‑driven test automation (Part 3). It discusses how a Google team evaluated only half of the author’s argument, leaving the remaining portion unresolved. The post was originally published on the Towards Data Science platform.

By Gal Arav
Towards Data Science
Oct 22

Same Answer, Different Reason: What a Reasoning Model Says When It Follows a Planted Hint

The article examines how a small reasoning model responds when presented with incorrect answers to 200 exam questions. By inserting wrong answers and observing the model’s reasoning, the author explores whether the model’s justification aligns with the planted hint or diverges. The study highlights the model’s tendency to follow misleading cues and the implications for evaluating its reasoning process.

By Utkarsh Mangal
Towards Data Science
Oct 12

I Built a Trap for AI Image Judges. They Walked Into a Different One.

The article describes an experiment where the author created a trap for AI image judging systems. Three vision AI models failed to recognize the intended image, while GPT‑4o generated answers and Claude consistently responded with the letter 'B'. The piece highlights the differing capabilities and limitations of various AI models in image interpretation tasks.

By Abdullahi Dattijo
Towards Data Science
Oct 18

Both Charts Are Right. So Why Do They Disagree?

The article discusses a statistical paradox illustrating how combining data can reverse the narrative presented by individual charts. It explains that both charts can be correct yet appear contradictory when aggregated. The piece highlights the importance of careful data integration to avoid misleading conclusions.

By Sara A. Metwalli
Towards Data Science
Oct 27

An AI Agent Can Spot a Mistake and Still Not Fix It

The article describes an experiment where an AI agent was able to identify incorrect model decisions while interacting with a web page, yet failed to correct the errors. By replaying correct decisions through four intentionally flawed browser actions, the author demonstrates that a simple page check missed half of the erroneous clicks. This illustrates that detecting a fault and successfully recovering from it are distinct capabilities.

By Abdullahi Dattijo
Towards Data Science
Oct 17

The Number Your Quantization Gate Doesn't Have

The article discusses the absence of a threshold‑dependent metric across ten quantization guides, highlighting what this omission reveals and introducing two numbers that address the gap. It examines how the lack of such a metric obscures certain insights in quantization practices. The piece offers a solution by presenting two specific figures that fill this missing analytical space.

By Maxim Podstrechnyy