An AI agent passed every metric in the eval harness I published, then the CFO killed it — its successful resolutions cost more than the humans it replaced. The one metric that predicts whether an agent survives production, and how to measure it without a rebuild.
Budgets for AI tokens can’t be infinite, no matter how much hyperscalers wish they were The post Drilling Into AI’s Financial Sustainability appeared first on Towards Data Science .
What it takes to turn counterfactual analysis into an AI product The post How I Built a Multi-Agent System for Interrupted Time Series Analysis (ITSA) appeared first on Towards Data Science.
The article explains how the five core assumptions of MLOps monitoring are violated when agents are deployed to production, leading to inherited signals that incorrectly mark failed runs as healthy. It highlights the specific ways in which agent-based systems disrupt traditional monitoring stacks and the implications for reliability and performance. The piece serves as a warning for practitioners transitioning from MLOps to AgentOps, outlining the critical monitoring gaps that arise.
The article discusses how agentic AI is reshaping the analytics stack by taking over more execution tasks. It raises the question of which responsibilities should remain with human analysts versus AI agents and explores the importance of this distinction. The piece highlights the evolving role of AI in analytics and the need to define clear boundaries between human and machine work.
A practical tutorial for recording model tool requests, real function results, patches, checks, screenshots, and a saved run log. The post How to Debug AI Coding Agents When They Change the Wrong Thing appeared first on Towards Data Science .
How to set the rules that keep agents effective and out of trouble The post What AI Agents Should Never Do on Their Own appeared first on Towards Data Science .
Morgan Stanley uses AI evals to shape the future of financial services
Learn how enterprises can manage AI investments in the agentic era by measuring useful work per dollar, improving efficiency, and scaling high-value workflows.
A team cut their AI inference bill by more than half. Three months later, customer satisfaction was dropping and the cost savings were tied to the quality loss.
Morgan Stanley uses AI evals to shape the future of financial services
One near miss, four months of running agents, and the question almost nobody is asking: what are you supposed to do while the AI writes the code? The post AI Made Me 5x Faster. It Also Made Me 5x Wors...