Table of Contents

Part 1: The Architecture

• Chapter 1: Why CPUs Can't Have 10,000 Cores
• Chapter 2: Why a GPU multiplies matrices faster
• Chapter 3: What "parallelizable" actually means

Part 2: Feeding the Beast

• Chapter 4: Why GPUs care so much about memory bandwidth
• Chapter 5: Compute-bound vs. memory-bound
• Chapter 6: The GPU memory hierarchy

Part 3: Inside the Silicon

• Chapter 7: What problem do warps solve?
• Chapter 8: Why branching is bad on GPUs
• Chapter 9: What is SIMT? (and how it differs from SIMD)
• Chapter 10: Why occupancy matters

Part 4: Putting it Together

• Chapter 11: Why AI runs on GPUs but databases don't
• Chapter 12: What is latency hiding
• Chapter 13: What if a GPU had CPU-style caches
• Chapter 14: The memory wall & HBM
• Chapter 15: Designing an LLM training processor

The Translation Barrier

• Chapter 16: Why C++ won't auto-compile to GPU
• Chapter 17: A kernel is not a function
• Chapter 18: Why thread indexing exists

The Hardware Hierarchy

• Chapter 19: Why threads group into blocks
• Chapter 20: Why shared memory is fast
• Chapter 21: Why RAM isn't visible to the GPU
• Chapter 22: Kernel launch overhead

Synchronization & Hazards

• Chapter 23: What syncthreads protects
• Chapter 24: Why race conditions happen
• Chapter 25: What makes CUDA code "correct"

Scale & Performance

• Chapter 26: Why the GPU context switch is free but CPU's is slow
• Chapter 27: The step-by-step anatomy of a kernel launch
• Chapter 28: Why NVIDIA made CUDA asynchronous
• Chapter 29: Coalesced vs strided memory access
• Chapter 30: Why GPU programs actually slow down
• Chapter 31: Why matrix add scales but a linked list dies
• Chapter 32: When NOT to use a GPU — and the reduction tree

The Memory Wall

• Chapter 33: Your ALUs are idle. Memory is the real bottleneck.
• Chapter 34: What is arithmetic intensity? The currency of GPU optimization.
• Chapter 35: Two hard ceilings. One tells you to fix your math. The other says: fix your memory.

The Register Physics

• Chapter 36: Why not build a GPU entirely from registers?
• Chapter 37: Where should data live? The register-occupancy tradeoff.
• Chapter 38: What is occupancy, really? It's a ratio, not a count — and 100% is not the goal.
• Chapter 39: Why higher occupancy can make performance worse.

Register Spilling

• Chapter 40: What is register spilling? The cliff between 1 ms and 50 ms.
• Chapter 41: Splitting the giant
• Chapter 42: When spilling is acceptable: and when it destroys performance.

Global Memory & Coalescing

• Chapter 43: Why global memory latency is 350 to 1,000 cycles: the physics of distance.
• Chapter 44: What is memory coalescing? 32 threads, 1 transaction vs 32 transactions.
• Chapter 45: DRAM row-buffer thrashing: why scattered access is structurally broken.

Shared Memory & Tiling

• Chapter 46: What is predication? Why GPUs refuse to branch.
• Chapter 47: Compare-and-swap: The universal atomic primitive.
• Chapter 48: Register tiling: The tile inside the tile.

GEMM & The Synthesis

• Chapter 49: Wave quantization: the hidden cost of the last wave
• Chapter 50: The traffic equation: throughput, latency, and concurrency
• Chapter 51: The second lever: hiding latency within a single thread
• Chapter 52: Measure, don't guess: the top-down profiling discipline

The Measurement Mandate

• Chapter 53: The 91 missing TFLOPS
• Chapter 54: Observe, don't assume
• Chapter 55: Nsight Systems vs Compute
• Chapter 56: What kernel time measures