An AI agent passed every metric in the eval harness I published, then the CFO killed it — its successful resolutions cost more than the humans it replaced. The one metric that predicts whether an agent survives production, and how to measure it without a rebuild.
Budgets for AI tokens can’t be infinite, no matter how much hyperscalers wish they were The post Drilling Into AI’s Financial Sustainability appeared first on Towards Data Science .
A practical tutorial for recording model tool requests, real function results, patches, checks, screenshots, and a saved run log. The post How to Debug AI Coding Agents When They Change the Wrong Thing appeared first on Towards Data Science .
How to set the rules that keep agents effective and out of trouble The post What AI Agents Should Never Do on Their Own appeared first on Towards Data Science .
Morgan Stanley uses AI evals to shape the future of financial services
Learn how enterprises can manage AI investments in the agentic era by measuring useful work per dollar, improving efficiency, and scaling high-value workflows.
A team cut their AI inference bill by more than half. Three months later, customer satisfaction was dropping and the cost savings were tied to the quality loss.
Morgan Stanley uses AI evals to shape the future of financial services
Learn how evals help businesses define, measure, and improve AI performance—reducing risk, boosting productivity, and driving strategic advantage.
arXiv:2607. 02197v1 Announce Type: cross Abstract: The society and emerging risk-based regulatory frameworks for AI underscore the need for rigorous risk assessment to ensure safe and reliable AI systems.
Map AI value, design workflows, redefine talent, upgrade the executive team, and measure the business impact. The post Redesign Work Before You Add More AI Agents appeared first on Towards Data Science .
arXiv:2606. 08296v1 Announce Type: new Abstract: A key premise in leading arguments for existential risk from artificial intelligence is that malfunctioning artificial agents could not be easily shut down.
arXiv:2606. 16465v1 Announce Type: new Abstract: AI agents can now take irreversible actions in operational systems, but agent-caused losses are still not clearly assigned, priced, or transferred.