Why AI runs on GPUs but databases don't

*worked out from first principles*

A GPU can chew through billions of numbers a second. So a fair question is why we don't just run everything on it. Why is Google's search, a bank's ledger, Amazon's inventory, all of it, still crawling along on CPUs.

The answer says a lot about what a GPU actually is, because the same machine that is perfect for AI is genuinely terrible for a database. And the reason isn't power. It's the shape of the work.

What AI Actually Is Underneath

Start with what AI actually is underneath, because it isn't what most people picture.

An AI model is not a giant pile of if/else logic. Strip it down and it's just math, enormous amounts of one kind of math. When you hand a model a photo of a cat, it doesn't see a cat. It sees a big grid of numbers, the pixel values. The model carries its own grid of numbers it learned during training, called weights. And the entire act of thinking is just multiplying the input grid against the weight grid, over and over, layer after layer.

Data flows in one end and gets multiplied forward through the network. That's the whole engine.

Why AI and GPUs are a Perfect Match

Now hold that next to how a GPU is built, and you'll see why they're made for each other. Three reasons.

• First, the work is pure matrix multiplication, and a GPU is physically a grid of thousands of tiny cores. Multiplying two large grids breaks into millions of tiny multiplications that don't depend on each other, and a GPU has exactly the army to do them all at once. The math and the machine have the same shape.
• Second, modern GPUs carry special hardware built just for this, called Tensor Cores. A normal core multiplies one pair of numbers at a time. A tensor core swallows a whole small grid and multiplies it in basically one shot, and it's hardwired for the compact number formats AI uses. It's a chunk of silicon that exists for no reason other than to do this one operation faster.
• Third, and this is the quiet one, AI has no surprises in its path. The data marches forward through the network and the corrections flow backward, and there's no point where the work suddenly asks a question and forks. So all 32 threads in a group stay locked in step doing the same multiply. No branching, no team splitting, none of the stalling that kills GPU performance. It's an army on a parade march, every step identical.

Why Databases Hit a Wall

Now flip to a database, and every one of those advantages turns into a wall.

1. Low Arithmetic Intensity: The math is trivial but the data is huge. A database mostly looks things up and adds things. Find this name. Total this column. Add 100 to this balance. The actual computation is nothing, but it has to drag mountains of data across the chip to do it. So a GPU core finishes its tiny bit of math instantly and then sits there starving, waiting on data that trickles in over the wires. This is the low arithmetic intensity problem, and it's poison for a GPU.
2. Branching and Divergence: A database is nothing but branching. Every real query is full of conditions. "Show me black t-shirts under 500 rupees" means checking, for every single row, is it a t-shirt, is it black, is it under 500. Each row takes a different decision. And a group of GPU threads forced down different decision paths is exactly the divergence disaster. Half the threads freeze while the other half run, then they swap. A query packed with WHERE and JOIN conditions makes the GPU's threads constantly split and stall against each other.
3. Random Access: The data is scattered. AI data arrives in neat ordered rows the GPU can stream. Database data lives anywhere in memory, fetched in a random, jumping pattern. A GPU wants its data lined up and predictable, and random access turns the supply road into a jam.
4. ACID Locking: This is the deepest one. Databases demand order. There's a rule called ACID, and the core of it is that transactions must happen safely, one at a time when they touch the same thing. Picture you and your brother both trying to withdraw 500 rupees from the same account with 500 in it, at the same moment from two ATMs. If both go through, the bank just lost money. So the database locks the account, lets one withdrawal finish completely, and only then lets the next one in. Now imagine handing that careful, step-by-step processing to a GPU swarm that pounces on data all at once. Who enforces the lock? You want one strict manager handling each transaction in order, safely, with low delay. That is exactly what a CPU is.