Towards Data Science
Jun 7
Building a Multi-Agent System in Python
An introduction to multi-agent systems The post Building a Multi-Agent System in Python appeared first on Towards Data Science .
Why Most Multi-Agent Systems Fail Even When Evaluation Passes
The article explains how to detect a payload that appears correct yet is not, by employing a watchdog pattern in Python. It discusses the challenges that cause many multi‑agent systems to fail even when their evaluations succeed. The post was originally published on Towards Data Science.
Related stories
Towards Data Science
Aug 31
AgentOps Is Not MLOps: What Breaks in Your Monitoring Stack When Agents Go to Production
The article explains how the five core assumptions of MLOps monitoring are violated when agents are deployed to production, leading to inherited signals that incorrectly mark failed runs as healthy. It highlights the specific ways in which agent-based systems disrupt traditional monitoring stacks and the implications for reliability and performance. The piece serves as a warning for practitioners transitioning from MLOps to AgentOps, outlining the critical monitoring gaps that arise.
Towards Data Science
Aug 6
I Built a Tool-Calling Agent in Python. Here’s How I Debugged It
A minimal loop with real API calls, validation, compact outputs, and trace evidence before adding an agent framework The post I Built a Tool-Calling Agent in Python. Here’s How I Debugged It appeared first on Towards Data Science .
Towards Data Science
Oct 25
Agentic AI in Production: Why Your Agent Keeps Answering After Its Tools Fail
The article discusses how an agent can continue to produce confident answers even when its tools fail, leading to a misleadingly positive dashboard display. It highlights the issue of agents generating responses from nothing after tool breakdowns, which can create a false sense of reliability. The piece serves as a cautionary note for developers deploying agentic AI systems.
Hugging Face Trending Papers
3d ago
Signals: Continuous Diagnostics of AI Agents
AI agents can make incorrect decisions or waste resources even when their tools and infrastructure work normally. Their nondeterministic behavior makes these problems difficult to reproduce and diagno...
arXiv AI
Oct 15
Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
arXiv:2610.18579v1 Announce Type: new Abstract: System One models choose an agent's next action by scoring a supplied set of options in a single pass. This makes them fast and cheap, but when they ch...
arXiv AI
1d ago
Signals: Continuous Diagnostics of AI Agents
The paper "Signals: Continuous Diagnostics of AI Agents" introduces a method for diagnosing problems in AI agents by analyzing agentic traces. It defines 45 problem types across eight families, categorizes findings into engineering issues called Signals, and assesses their severity. The method was evaluated on public agent executions, achieving a 91.43% F1 score in matching Signals to reference issues, outperforming a one-pass LLM control by about 18 percentage points.
Towards Data Science
Sep 17
How I Built a Multi-Agent System for Interrupted Time Series Analysis (ITSA)
What it takes to turn counterfactual analysis into an AI product The post How I Built a Multi-Agent System for Interrupted Time Series Analysis (ITSA) appeared first on Towards Data Science.
arXiv AI
Aug 20
Position: Behavioral Systems Require Behavioral Tests
The paper argues that artificial agentic systems, which operate as behavioral systems by interacting with dynamic environments, pursuing goals, and adapting over time, should be evaluated through systematic observation, perturbation, and interpretation of their actions rather than solely on performance outcomes. It draws on lessons from behavioral sciences to motivate this position and proposes a research agenda that includes methods for recovering decision strategies from action sequences, constructing environments that isolate behavioral differences, and probing emergent dynamics in multi‑agent systems. These directions aim to establish a rigorous science of AI behavior.
arXiv Machine Learning
Sep 17
Locating Hidden Failures Makes Long-Horizon Agents More Reliable
The paper introduces Traverse, a benchmark of 2,518 agent trajectories and 6,967 annotated mistakes across software engineering, computer use, and science tasks, revealing that failures often go unrecovered and can cause irreversible harm before a run is deemed successful. It shows that human judges struggle to detect the first mistake in most runs, while a 4‑billion‑parameter verifier called Scout can locate failures more effectively and improve task success when used to select among candidate runs. The study demonstrates that making failure detection inexpensive and reliable can enable long‑horizon agents to learn from their own mistakes and increase trustworthiness in autonomous AI.
Towards Data Science
Oct 27
An AI Agent Can Spot a Mistake and Still Not Fix It
The article describes an experiment where an AI agent was able to identify incorrect model decisions while interacting with a web page, yet failed to correct the errors. By replaying correct decisions through four intentionally flawed browser actions, the author demonstrates that a simple page check missed half of the erroneous clicks. This illustrates that detecting a fault and successfully recovering from it are distinct capabilities.
arXiv AI
Jun 30
Preventing Error Propagation in Multi-Agent AI through Runtime Monitoring
arXiv:2606. 29026v1 Announce Type: new Abstract: Multi-agent AI systems can improve answer selection by allowing different language models to exchange reasoning traces, revise initial predictions, and support a final decision.