Here is a fact that should startle anyone who has studied CPUs: nobody outside NVIDIA knows the full instruction set of an NVIDIA GPU. There is no public manual for the machine code; it changes — sometimes drastically — with every hardware generation; and yet a CUDA program compiled in 2010 still runs on a GPU designed in 2026. On the CPU side, where the ISA is the sacred contract that freezes x86 opcodes for forty years, this would be unthinkable.

The GPU world pulled off a beautiful trick: it kept the idea of the ISA-as-contract but moved the contract up one level — from the silicon to a virtual instruction set that no chip has ever executed directly. This lesson follows a kernel from source code to executing silicon, and the journey explains both how GPU vendors redesign their machines every two years without breaking anything, and why we will be free to invent our own ISA when this course builds its capstone GPU in module 12.

Two ISAs: the one you target, and the one that runs

At build time, the compiler translates your kernel not into machine code but into a virtual ISA — NVIDIA's is called PTX — a clean, stable, assembly-like intermediate language for an idealised SIMT machine:

This is what ships inside your application, and this is the contract: PTX is documented, versioned, and forward-compatible. Then, on the user's machine, the driver JIT-compiles the PTX into the GPU's actual machine ISA — NVIDIA's is called SASS — which is undocumented, generation-specific, and free to change utterly. New register file layout? Different instruction encodings? Split an instruction into three, fuse five into one? Fine: ship a new driver, and every existing program JIT-compiles onto the new silicon. The hardware team got the one freedom CPU architects dream of: redesign the microarchitecture and the machine ISA without breaking a single binary.

The whole journey, drawn

Step through the pipeline. Notice where the dashed line falls: everything above it happens once, on the developer's machine; everything below happens on every user's machine, against whatever GPU happens to be installed:

The launch path: kernels as batch jobs

Now follow one kernel launch. Your host program makes an API call — conceptually launch(vecAdd, blocks, threadsPerBlock, args). That call does not reach into the GPU and start threads. Instead:

• the driver writes a launch packet (which kernel, grid shape, argument pointers) into a command queue in memory — and returns immediately: launches are asynchronous;
• the GPU's command processor — a small dedicated front-end processor on the chip — reads the queue, parses the packet, and sets up the launch;
• a hardware work distributor hands blocks to SMs as space frees up, until the grid is drained.

Notice what is missing: there is no operating system on the GPU. No scheduler process, no interrupts driving time-slices, no syscalls. A kernel is a batch job: submitted into a queue, run to completion, results collected afterwards — 1960s mainframe operation reborn at gigahertz. Every OS-like duty (scheduling queues, memory management, compilation, multiplexing between applications) is performed by the driver on the host. The driver is the GPU's operating system.

The memory side of the contract

The runtime's other half manages memory. The GPU has its own DRAM, so the classic discipline is explicit: allocate device memory, copy inputs across the interconnect, launch, copy results back. Those transfers ride a bus that is orders of magnitude narrower than the GPU's local memory bandwidth, which is why "keep data resident on the device" is the first commandment of GPU performance. Modern stacks soften the ceremony with unified memory — one pointer space, pages migrated on demand by driver and hardware — a convenience whose costs and machinery we will weigh when we design the memory system in the next module.

Why a hardware course cares

This lesson looks like software, but it is load-bearing for the silicon we design later. First, the launch path is hardware: our capstone GPU in module 12 needs a command processor — something that reads launch packets from memory and feeds a work distributor — or nothing ever runs. Second, and more liberating: the virtual/physical ISA split means the machine ISA is ours to invent. As long as some translation layer accepts kernels above us, we can choose our own instruction encodings, register conventions and scheduling rules, optimised purely for clean hardware — exactly the freedom NVIDIA exercises every generation, and exactly what we will do.