The Roofline Model
You've tuned a kernel for a
GPU
that can do a trillion floating-point operations a second, and you're getting a tenth of that. Should you
add more arithmetic tricks, or is something else the bottleneck? Guessing is expensive. The
roofline model answers it on the back of a napkin: it plots the maximum
performance any kernel can reach on a given machine, and tells you at a glance whether you're limited by
the machine's compute or by its memory bandwidth — and therefore what
to optimise.
It's called a roofline because the performance ceiling looks like the roof of a house: a slanted eave on
the left (you're memory-bound) meeting a flat ridge on the right (you're compute-bound). Where your kernel
sits under that roof tells you the whole story.
The one number that decides everything: arithmetic intensity
The key idea is arithmetic intensity (also "operational intensity"): the number of
floating-point operations a kernel does per byte it moves from memory.
I \;=\; \frac{\text{FLOPs performed}}{\text{bytes moved from memory}} \quad\left[\frac{\text{FLOP}}{\text{byte}}\right].
It measures how much useful work you extract from each byte you fetch. A vector add
c_i = a_i + b_i reads 8 bytes, writes 4, and does 1 FLOP — intensity
\approx 1/12, dreadful. A dense matrix multiply reuses each loaded value
N times, so its intensity grows with the problem size — it can reach hundreds.
Intensity is a property of the algorithm, not the machine, and it's the x-axis of the roofline.
The roofline equation
A machine has two hard limits: a peak compute rate
\pi (FLOP/s) and a peak memory bandwidth
\beta (bytes/s). A kernel of intensity I can, at
best, be fed FLOPs at rate I \times \beta (intensity times how fast bytes
arrive), but never beyond the machine's raw compute peak \pi. The attainable
performance is the smaller of the two:
P_{\text{attainable}} \;=\; \min\big(\;\pi,\;\; I \times \beta\;\big).
Plot P against I and you get the roof: a straight
line of slope \beta rising from the origin (the memory
roofline), which flattens out the instant it hits the horizontal compute
roofline at height \pi.
- arithmetic intensity I = \text{FLOPs} / \text{bytes} is
a property of the algorithm;
- attainable performance is P = \min(\pi,\; I\,\beta) — the lower of the
compute roof \pi and the memory roof
I\,\beta;
- the two meet at the ridge point
I^\star = \pi / \beta: below it you're memory-bound, above it
compute-bound.
Read the roof
Below is a live roofline. The rising line is the memory roof (slope = bandwidth
\beta); the flat line is the compute roof (height = peak
\pi). A kernel's attainable performance is the lower of the two at its
intensity. Drag the bandwidth slider and watch the sloped roof tilt — a machine with more bandwidth pushes
its ridge point left, so more kernels become compute-bound. Drag peak compute and the flat roof
rises, moving the ridge right. The ridge point I^\star = \pi/\beta is
where they cross.
The practical reading: a kernel to the left of the ridge is memory-bound
— it will never reach peak FLOPs no matter how you tune the arithmetic; the only cure is to move fewer
bytes (better cache reuse, blocking) or raise its intensity. A kernel to the right is
compute-bound — now, and only now, do vectorization, better instruction scheduling, and
tensor cores actually pay off.
Worked example: where does SAXPY land?
Take a machine with peak \pi = 1000\ \text{GFLOP/s} and bandwidth
\beta = 100\ \text{GB/s}. Its ridge point is
I^\star = 1000/100 = 10\ \text{FLOP/byte}. Now SAXPY
(y_i = a x_i + y_i) moves 12 bytes per element (read
x, read and write y) and does 2 FLOPs (a multiply and
an add), so its intensity is I = 2/12 \approx 0.17. That's far to the left of
the ridge, so
P = \min(1000,\; 0.17 \times 100) = \min(1000,\; 17) = 17\ \text{GFLOP/s}.
Just 1.7\% of peak — and utterly hopeless to fix by speeding up the arithmetic.
The roofline told us in one line: SAXPY is memory-bound; buy bandwidth or raise intensity, don't touch the
ALU. Run the numbers for a spread of kernels:
// Roofline: attainable = min(peak, intensity * bandwidth).
const peak = 1000; // GFLOP/s (pi)
const bw = 100; // GB/s (beta)
const ridge = peak / bw; // FLOP/byte where memory roof meets compute roof
function attainable(intensity: number): number {
return Math.min(peak, intensity * bw);
}
console.log(`peak = ${peak} GFLOP/s, bandwidth = ${bw} GB/s`);
console.log(`ridge point = ${ridge} FLOP/byte`);
console.log("");
const kernels = [
{ name: "SAXPY (a*x+y)", I: 2 / 12 },
{ name: "stencil / blur", I: 0.5 },
{ name: "sparse matvec", I: 0.25 },
{ name: "dense matmul (blocked)", I: 40 },
];
for (const k of kernels) {
const p = attainable(k.I);
const bound = k.I < ridge ? "MEMORY-bound" : "COMPUTE-bound";
const pct = ((p / peak) * 100).toFixed(1);
console.log(`${k.name}: I=${k.I.toFixed(2)} -> ${p.toFixed(0)} GFLOP/s (${pct}% of peak, ${bound})`);
}
Using it to guide optimisation
The roofline turns "make it faster" into a decision tree. Find your kernel's intensity, drop it on the
roof, and:
| Where you are | What limits you | What to do |
|---|
| Left of ridge (low I) | memory bandwidth | move fewer bytes: cache blocking, data reuse, better layout — raise intensity |
| Right of ridge (high I) | compute peak | vectorize, use FMA / tensor cores, improve scheduling |
| On the roof | the machine itself | you've won on this machine — need a bigger one |
| Well below the roof | something else (stalls, poor parallelism) | fix that before anything else |
That last row matters: a real kernel usually runs below its roofline, and the gap is your
opportunity. The roof is the ceiling; closing the distance to it is the work.
Real machines span a wild range — intensities from 0.1 to hundreds, and
performance across orders of magnitude — so practitioners plot the roofline on log-log
axes. There the sloped memory roof becomes a straight 45° line (a power law
P = I\beta is linear in log-log), the flat compute roof stays horizontal, and
a whole zoo of kernels fits on one readable chart. You can even stack multiple roofs — one for
DRAM, one for L2, one for L1 bandwidth, plus separate compute roofs with and without vectorization — so
the picture shows every ceiling a kernel might hit as you optimise it. Our linear plot here is the same
idea, just zoomed in near the ridge where the geometry is easiest to see.
The denominator is the traffic to the level of memory you're bounded by — usually bytes actually fetched
from DRAM — not the size of your data structure. Caching changes it: if a value is loaded once and
reused from cache a hundred times, those reuses cost no DRAM traffic, so the effective intensity
is far higher than a naïve "bytes in the array" count suggests. This is exactly why cache blocking works —
it doesn't do less arithmetic, it moves fewer bytes, sliding the kernel rightward along the roof
from memory-bound toward compute-bound. Count the traffic that actually crosses the bottleneck, or the
roofline will mislead you.