Towards Data Science

An AI Agent Can Spot a Mistake and Still Not Fix It

The article describes an experiment where an AI agent was able to identify incorrect model decisions while interacting with a web page, yet failed to correct the errors. By replaying correct decisions through four intentionally flawed browser actions, the author demonstrates that a simple page check missed half of the erroneous clicks. This illustrates that detecting a fault and successfully recovering from it are distinct capabilities.

Towards Data Science
Oct 27

I Broke a Browser Agent on Purpose. Spotting the Fault Was Not the Same as Fixing It

The article describes an experiment where the author intentionally broke a browser agent by replaying correct model decisions through four faulty browser actions. A basic page check failed to detect half of the incorrect clicks and incorrectly flagged every successful click within an embedded page. The piece highlights the challenges of diagnosing and resolving such browser-related faults.

By Abdullahi Dattijo
Towards Data Science
Oct 25

Agentic AI in Production: Why Your Agent Keeps Answering After Its Tools Fail

The article discusses how an agent can continue to produce confident answers even when its tools fail, leading to a misleadingly positive dashboard display. It highlights the issue of agents generating responses from nothing after tool breakdowns, which can create a false sense of reliability. The piece serves as a cautionary note for developers deploying agentic AI systems.

By Benjamin Nweke
Towards Data Science
Aug 27

Agentic AI Is Rewriting The Analytics Stack But There's One Skill It Still Can't Touch

The article discusses how agentic AI is reshaping the analytics stack by taking over more execution tasks. It raises the question of which responsibilities should remain with human analysts versus AI agents and explores the importance of this distinction. The piece highlights the evolving role of AI in analytics and the need to define clear boundaries between human and machine work.

By Rashi Desai
Towards Data Science
Oct 22

Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions

The article reports on an experiment where an AI agent was allowed to search for evidence whenever it chose. The agent discovered some papers that a standard search missed, but also lost some that the plain search had found, and overall the gains were insufficient to be trusted. The post examines whether agentic Retrieval-Augmented Generation (RAG) truly retrieves better evidence across 298 questions.

By Abdullahi Dattijo
Towards Data Science
Oct 16

I Built a Claim-Checker for Writing With Codex. It Passed a Wrong Claim.

The article describes a claim‑checker tool built for writing with Codex that successfully identified the correct passage but failed to detect a mismatch, illustrating the limitations of rule‑based checks. The audit highlights where these automated checks break down and provides insights into improving claim verification processes.

By Benjamin Nweke
Towards Data Science
Aug 27

Stop Giving Your AI Agent a Search Box and Start Giving It Typed Tools, Hard Bounds, and a Gate It Cannot Talk Past

The article explores the effects of removing a search box from an AI agent and instead providing it with typed tools, hard bounds, and a gate that it cannot bypass. It examines how the agent navigates a knowledge graph within strict limits and discusses findings from four models and one incorrect prediction regarding the value of this approach.

By Miodrag Cekikj