The Texture Unit

In colour and texture you learned what a texture means: an image glued to a surface, sampled at a coordinate (u, v). In shader code that is one line — texture(sampler, uv) — and it is the most common line in real-time graphics: a modern frame issues billions of samples. So ask the module's standing question: what silicon answers it? Not the shader core. The sample is handed to a dedicated texture unit, a fixed-function pipeline sitting beside the SMs, and what it does for that one innocent call is a small saga:

Address maths: turn (u,v) \in [0,1]^2 into texel coordinates, applying the wrap mode — repeat (take the fraction), clamp to the edge, or mirror.
Level-of-detail: measure how big a screen pixel is in texels and choose a mip level.
Fetch: read eight texels — a 2\times2 neighbourhood from each of two mip levels — through its own cache.
Filter: blend them with seven linear interpolations and return one colour.

Done in shader code, that saga is dozens of instructions per sample. Hardwired, it is a pipeline that answers one filtered sample per clock — which is why it has been fixed-function silicon since the very first consumer GPU, and still is.

Choosing the mip: the 2×2 quads pay off

Why mipmapping at all? Picture a textured floor stretching to the horizon. Up close, one screen pixel covers less than one texel — magnification, easy. Far away, one screen pixel covers hundreds of texels; sampling just one of them (whichever lands under the pixel centre) picks an effectively random texel, and as the camera moves the pick changes every frame — the distant floor shimmers and crawls with aliasing. The honest fix is to average all the texels under the pixel, but that is hopeless at one sample per clock.

The mipmap chain is that average, precomputed: level 0 is the full image, level 1 half the size (each texel the average of 4), level 2 a quarter, down to a single texel — all for one-third extra memory. Now "average hundreds of texels" becomes "read the level where one texel ≈ the pixel's footprint". The unit computes the footprint from the screen-space derivatives of u and v — and here the rasterizer's 2\times2 quads pay off: differencing neighbouring quad lanes gives the derivatives almost for free. If the footprint spans f texels, the level is

\lambda = \log_2 f \qquad \text{(footprint 8 texels } \Rightarrow \lambda = 3\text{)}

\lambda is rarely a whole number, so trilinear filtering samples the two bracketing levels (\lfloor\lambda\rfloor and \lfloor\lambda\rfloor + 1) and blends by the fraction — bilinear twice, plus one more lerp: 4 + 4 texels, 3 + 3 + 1 = 7 lerps, every sample.

The seven lerps, drawn

Step through one trilinear sample. Texel values are shown at their centres; watch four texels collapse to one bilinear value, twice, and the two levels blend into the answer.

Run the filter yourself

Bilinear filtering on a tiny checkerboard — the four fetches, the four weights, and the mip-level arithmetic, in code you can poke at.

// A 4×4 one-channel texture: a checkerboard of 0s and 255s. const W = 4, H = 4; const texel = (x: number, y: number): number => { x = Math.min(W - 1, Math.max(0, x)); // clamp addressing at the edges y = Math.min(H - 1, Math.max(0, y)); return ((x + y) % 2) * 255; }; function bilinear(u: number, v: number): number { // Address maths: uv in [0,1] → continuous texel coords (centres at half-integers). const xf = u * W - 0.5, yf = v * H - 0.5; const x0 = Math.floor(xf), y0 = Math.floor(yf); const fx = xf - x0, fy = yf - y0; // The four fetches and their weights. const t00 = texel(x0, y0), t10 = texel(x0 + 1, y0); const t01 = texel(x0, y0 + 1), t11 = texel(x0 + 1, y0 + 1); const w00 = (1 - fx) * (1 - fy), w10 = fx * (1 - fy); const w01 = (1 - fx) * fy, w11 = fx * fy; console.log(` texels ${t00} ${t10} ${t01} ${t11} weights ` + [w00, w10, w01, w11].map((w) => w.toFixed(2)).join(" ")); // Three lerps' worth of arithmetic, as a weighted sum. return t00 * w00 + t10 * w10 + t01 * w01 + t11 * w11; } for (const [u, v] of [[0.30, 0.30], [0.42, 0.30], [0.50, 0.50]]) { console.log(`sample (${u}, ${v}):`); console.log(` result ${bilinear(u, v).toFixed(1)}\n`); } // LOD selection: footprint of f texels → mip level log2(f). console.log("footprint → mip level (trilinear blends floor and floor+1):"); for (const f of [1, 2, 3, 8, 20]) { const lod = Math.max(0, Math.log2(f)); console.log(` ${f} texels → λ = ${lod.toFixed(2)} → levels ` + `${Math.floor(lod)} and ${Math.floor(lod) + 1}, fraction ${(lod - Math.floor(lod)).toFixed(2)}`); }

Notice the middle sample: at (0.5, 0.5) all four weights are 0.25 and the checkerboard averages to exactly 127.5 — bilinear filtering is a little 4-texel average, slid smoothly around the image.

Anisotropy: when the footprint isn't square

Trilinear assumes a pixel's footprint in texel space is roughly square. Look along a floor at a grazing angle and it isn't — it is a long thin ellipse, perhaps 2 texels wide and 16 long. Trilinear must pick one level for both directions: match the short axis and the long axis aliases; match the long axis and everything blurs (the classic smeared-road look). Anisotropic filtering instead takes several trilinear probes spaced along the footprint's major axis — up to 16 of them at "16×" — and averages, sharp in one direction and smooth in the other. The cost is real (each probe is a full 8-texel trilinear sample), which is why it has a quality dial, and why the texture unit's ability to sustain it comes from the next idea.

Feeding the beast: texture caches and Morton order

Eight texels per sample, billions of samples — the arithmetic is the easy half. The hard half is memory. Texture units survive on two locality tricks:

A dedicated texture cache per unit (backed by the GPU's L2). Neighbouring pixels sample neighbouring texels, and bilinear neighbourhoods overlap heavily — a texel fetched for one pixel is almost always needed again by the next. Small cache, enormous hit rate.
Tiled ("Morton-order") storage. A row-major image is poison for 2-D access: the texel above yours is a whole row away in memory, so a cache line holds a useless 1-D strip. GPUs store textures in small 2-D tiles instead, commonly ordered along a Z-shaped Morton curve — so one cache line holds a little square of texels, and a bilinear's 2\times2 neighbourhood usually lives in one or two lines instead of four.

Block compression: decompressed inside the unit

The other bandwidth lever is to shrink the texels themselves. GPU formats (BC on desktop, ASTC on mobile) are fixed-rate block compressors: every 4\times4 texel block compresses to the same number of bytes — BC1 packs a 48-byte RGB block into 8 bytes (6:1), BC7 packs 64 bytes of RGBA into 16 (4:1). Fixed-rate is the whole point: the address of block (i, j) is a single multiply-add, so the unit can fetch just the block it needs and a tiny decoder inside the texture unit unpacks it on the fly. The texture stays compressed in memory and in the cache — every level of the hierarchy carries 4–8× more texels for the same bytes.

This is also why you cannot feed the unit a JPEG: JPEG is variable-rate (each block compresses to however many bits it happens to need), so there is no arithmetic that maps texel coordinates to a memory address — you'd have to decode the whole image first, which is exactly the random access a texture sampler cannot live without. Lossy-but-fixed-rate beats better-but-variable-rate, because the consumer is hardware.

one sample = address maths (wrap/clamp) + LOD from quad derivatives + 8 texel fetches + 7 lerps (trilinear), as a fixed-function pipeline;
mip levels are precomputed averages (⅓ extra memory) that replace per-sample averaging; anisotropic filtering adds probes along an elongated footprint;
bandwidth survives on texture caches + Morton-tiled layout (2-D locality) and fixed-rate block compression (BC/ASTC) decoded inside the unit — random-access-friendly, unlike JPEG;
a modern GPU carries hundreds of these units, several per SM.

A texture unit is a deep pipeline: a sample takes hundreds of cycles to come back, but the SM hides that by keeping many samples in flight and switching warps while they wait — as long as the addresses are known up front. A dependent texture read — computing a UV from the result of a previous sample, as in texture(B, texture(A, uv).xy) — breaks that: the second request cannot even be issued until the first returns, so the two latencies add instead of overlapping. One level of indirection is common and survivable (that's how lookup-table effects work); chains of them serialise the very latency all this machinery exists to hide. Keep your UV maths shallow, and let the compiler hoist sample addresses early.

How much of a GPU is texture hardware? Each SM carries its own cluster of texture units (typically four), so a big desktop chip ships several hundred of them, together answering on the order of half a trillion filtered samples per second. Historically the ratio ran the other way: the 1996 Voodoo Graphics was essentially a texture unit with a card attached — texturing was the whole product, and shading was an afterthought bolted around it. Thirty years on, the balance of the die has swung to programmable SMs, but the texture unit itself never became software: filtering's arithmetic is settled, its throughput demand is insatiable, and — as the first lesson of this module argued — that combination is exactly what silicon specialisation is for. The one line of shader code you write most often is the one the shader never executes.