Towards Data Science
Jul 8
The Real Challenge Limiting AI Models Today
Hint: it is not GPU speed! The post The Real Challenge Limiting AI Models Today appeared first on Towards Data Science .
When GPU Utilization Lies: The Hidden Systems Problem Slowing Modern AI
Why “average utilization” lies about how full your GPUs really are The post When GPU Utilization Lies: The Hidden Systems Problem Slowing Modern AI appeared first on Towards Data Science .
Related stories
Towards Data Science
Jul 21
How Much of a Data Science Workflow Can Run on a GPU Today? Part 1: Accelerating Data Preparation
Exploring GPU acceleration with cuDF, cudf. pandas, and the Polars GPU Engine The post How Much of a Data Science Workflow Can Run on a GPU Today?
Towards Data Science
Jul 23
Why Adding More AI Agents Made Our System Slower
The hidden cost of asynchronous systems, how tiny CPU tasks quietly became our biggest bottleneck while scaling hundreds of LLM agents. The post Why Adding More AI Agents Made Our System Slower appeared first on Towards Data Science .
Towards Data Science
Jun 9
The Hardware That Makes AI Possible
CPUs, GPUs, TPUs, and NPUs The post The Hardware That Makes AI Possible appeared first on Towards Data Science .
Towards Data Science
Jun 14
GPU Time-Slicing for Concurrent LLM Agents on Kubernetes
A systems-level deep dive into the hidden microarchitectural costs of Kubernetes GPU time-slicing, and what it actually costs to co-locate Agentic AI workloads. The post GPU Time-Slicing for Concurrent LLM Agents on Kubernetes appeared first on Towards Data Science .
arXiv AI
Jun 2
How Much Progress Has There Been in NVIDIA Datacenter GPUs?
arXiv:2601. 20115v3 Announce Type: replace-cross Abstract: As the role of modern Graphics Processing Units (GPUs) becomes increasingly essential for several computing tasks, analyzing their past and current progress is paramount for determining future constraints on scientific research.
OpenAI Blog
May 16, 2018
AI and compute
We’re releasing an analysis showing that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3. 4-month doubling time (by comparison, Moore’s Law had a 2-year doubling period)[^footnote-correction].
Towards Data Science
Jun 15
The System Always Knows: Why Local Efficiency and System Performance Are Not the Same Problem
How local optimization in last‑mile delivery can quietly break the system The post The System Always Knows: Why Local Efficiency and System Performance Are Not the Same Problem appeared first on Towards Data Science .
Towards Data Science
Jun 19
GPU-Resident Top-K for Agentic RAG: I Built a CUDA Kernel So My Retrieval Step Would Stop Bouncing Off the GPU
The PCIe transfer latency is silently bottlenecking your agentic inference. Here is how building a custom device-resident vector search kernel bypasses the CPU to unlock deterministic microsecond tail latencies.
Towards Data Science
Jul 9
Behind the Scenes of Distributed Training and Why Your GPU Wiring Matters as Much as Your Strategy
A measured look at distributed training, from DDP and FSDP to the ZeRO stages in between, and why the wiring between your GPUs matters as much as the strategy you choose The post Behind the Scenes of Distributed Training and Why Your GPU Wiring Matters as Much as Your Strategy appeared first on Towards Data Science .
Towards Data Science
Jul 11
That Is Embarrassing: Why Frontier AI Still Makes Things Up, and What to Do About It
The best AI models still hallucinate. These hallucinations are sometimes funny, and sometimes cause actual damage.
Towards Data Science
Jul 1
Why Powerful ML Is Deceptively Easy — Part 2
The next leakage problem is not only temporal. It is spatial, structural, and coverage-related.