Diagram deck

Nvidia CUDA in 100 Seconds

Channel Fireship

1. CUDA at a Glance: From Graphics Chip to AI Workhorse

Maps Section 1, "What Is CUDA?": the 2007 NVIDIA platform that turned the GPU from a graphics-only engine into the parallel computer behind AI training.

Before CUDA

A GPU was a graphics appliance: its thousands of cores spent every frame drawing game scenes, and running general-purpose math on them meant fighting the graphics pipeline for scraps of compute.

1
2007 · NVIDIA
CUDA ships: a parallel computing platform Compute Unified Device Architecture opens the GPU to developers - built on the earlier research of Ian Buck and John Nickolls.
2
The kernel
Developers tap the GPU's raw power A CUDA kernel is a function that runs on the GPU. Instead of pixels, it can compute over large blocks of data - in parallel.
3
Deep learning
AI training unlocks The same hardware that drew game frames now trains deep neural networks - data scientists worldwide use CUDA for the most powerful machine learning models.
Takeaway

CUDA is the bridge from "GPU equals video games" to "GPU equals AI at scale" - one kernel launch at a time.

2. CPU vs. GPU: Two Kinds of Cores

Contrasts Section 2, "Why GPUs Are Built for Parallel Work": the same operation over millions of pixels runs faster on 16,000+ simple GPU cores than on 24 powerful CPU cores.

The problem

A 1080p frame is over 2 million pixels, and at 60 FPS every one may be recomputed each frame - a firehose of matrix multiplication and vector transforms that has to happen in parallel.

Generalist

CPU

Intel Core i9

24

cores

  • Each core is large and powerful.
  • Very different tasks, handled one at a time.
  • Versatile: runs whatever you throw at it.
Specialist

GPU

NVIDIA RTX 4090

16,000+

cores

  • Thousands of small, simple cores.
  • One operation, executed on every pixel at once.
  • Throughput measured in teraflops.
Takeaway

A GPU is not one faster brain - it is thousands of smaller ones running the same math at once. CUDA exists to point all of those cores at real computing problems.

3. Host and Device: The Five-Step CUDA Flow

Sequences Section 3, "How CUDA Works: Host, Device, and Execution": the handshake between the host CPU and the device GPU - kernel first, data over, launch, execute, results back. Step through it below.

Legend: Host · CPU Device · GPU Crosses the boundary
Step 1 of 5

Click any step to jump to it. Steps before the current one are complete.

4. Grid, Blocks, and Threads: Inside a Kernel Launch

Zooms into Section 4, "Building Your First CUDA Application": what add<<<1, 256>>> launches on the device - one block of 256 threads, each computing its own index.

Host code add<<<1, 256>>>(A, B, C); one block, 256 threads
Grid The whole launch. Demos launch one block; grids can hold many blocks and reach into multiple dimensions to mirror the data.
Block 0 256 threads, all running the same kernel
0 1 2 3 … 255

Every thread works out which element it owns - hover a term to see what it means:

index = blockIdx.x × blockDim.x + threadIdx.x
blockIdx.x

Which block the thread lives in. The demo launches one block, so this is 0 for every thread.

blockDim.x

How many threads each block holds. The launch asks for 256, so this is 256.

threadIdx.x

The thread's position inside its block: 0 to 255. Multiplying and adding yields a unique index per thread.

Each thread C[index] = A[index] + B[index];

256 threads run the kernel at once, each adding the pair of elements at its own index. cudaDeviceSynchronize() then pauses the host until every thread has finished.

Managed memory

cudaMallocManaged allocates A, B, and C so the CPU and GPU share them - the demo skips the explicit copy steps from the general flow and reads the result straight back on the host.

Takeaway

The triple-bracket configuration decides how parallel a launch is - one block of 256 threads here, but grids scale to many blocks. Getting that shape right is the core skill for optimizing multi-dimensional structures like deep-learning tensors.