Real-Time vs Offline

Two industries point cameras at virtual worlds, and both call what they do "rendering" — yet they live in almost incomparable universes. A Pixar film gives its renderer hours per frame and harnesses thousands of machines through the night. A game gives its renderer about sixteen milliseconds — the blink between two frames at 60 fps — and must finish, every single frame, on the one GPU in the player's console. Same goal (turn a 3-D scene into a picture), budgets separated by many orders of magnitude, and therefore utterly different bags of tricks.

This page maps the two worlds — offline (film / VFX) and real-time (games / interactive) — pins down why their time budgets force different techniques, and then follows the exciting part: how hardware ray tracing, denoising and ML upscaling are finally letting the two converge. The comparison table below is the thing to memorise.

Offline: correctness over speed

In film and visual effects, the picture must survive a cinema screen and a director who freezes it frame by frame. So offline renderers chase physical correctness: they simulate light the way it really behaves, tracing rays that bounce around the scene until they reach a light. This is path tracing, and it delivers full global illumination — soft shadows, colour bleeding between surfaces, realistic reflections and refractions — essentially for free, because it is just physics.

The price is time. Production renderers — Pixar's RenderMan, Arnold, V-Ray — commonly spend from many seconds to several hours per single frame, firing millions of rays per pixel to beat down noise. No one machine could finish a movie in a lifetime, so studios run a render farm: hundreds or thousands of computers each rendering different frames overnight. Quality is effectively unlimited — you buy more of it with more machine-hours.

Real-time: a hard budget, every frame

A game cannot render overnight — it renders while you play, responding to the controller now. That imposes a brutal, non-negotiable clock. At 60 fps the whole frame — geometry, lighting, shadows, effects, UI — must be done in \tfrac{1}{60}\,\text{s} \approx 16.7\ \text{ms}; at 30 fps you get about 33\ \text{ms}. Miss the budget and the game visibly stutters.

So real-time rendering was historically built on rasterization — a fast projection of triangles to pixels that says nothing about how light bounces — plus a toolbox of clever approximations that fake what path tracing computes directly:

Each trick trades a slice of correctness for a huge slice of speed — exactly the deal a real-time budget demands.

Different metrics, not just different speeds

It is tempting to say offline is "slow" and real-time is "fast", but the two optimise different things. A render farm cares about throughput: total frames per machine-hour across the whole farm — a frame that takes two hours instead of one is merely twice the electricity. A game cares about latency and, crucially, deterministic frame time: every frame must finish inside the budget, consistently. An average of 60 fps that occasionally spikes to 40 ms feels worse than a steady 50 fps, because the hitch is visible and the controls feel laggy. Offline can happily let one hard frame take ten times longer; real-time cannot.

The two worlds side by side

Offline (film / VFX)Real-time (games)
Time budgetSeconds to hours per frame~16 ms (60 fps) or ~33 ms per frame
Core techniquePhysically based path tracing (full GI)Rasterization + baked / screen-space approximations
HardwareRender farm — thousands of machines, overnightOne GPU, in the player's hands, right now
Priority metricThroughput; correctness over speedLatency + deterministic frame time
QualityEffectively unlimited (buy more machine-hours)Fixed by the frame budget; "good enough" to sell the illusion
RenderersRenderMan, Arnold, V-RayUnreal, Unity, in-house game engines

Read down the columns and the pattern is clear: offline spends time to buy correctness; real-time spends correctness to buy time.

Worked example: why film needs a farm

Let's make the budget gap concrete. A feature film runs at 24 frames per second. A two-hour movie is

2\ \text{h} \times 3600\ \tfrac{\text{s}}{\text{h}} \times 24\ \tfrac{\text{frames}}{\text{s}} = 172{,}800\ \text{frames}.

Suppose a heavy VFX shot averages one hour per frame to render. On a single machine that is 172{,}800 hours, which is

\frac{172{,}800\ \text{h}}{24\ \tfrac{\text{h}}{\text{day}} \times 365\ \tfrac{\text{day}}{\text{yr}}} \approx 19.7\ \text{render-years}.

Roughly twenty years on one computer — obviously impossible on a production schedule, which is precisely why studios split the frames across a farm of thousands of machines and finish in weeks. Now compare a game: it renders 60 frames every second. Where film may spend an hour on a frame, the game spends about 16.7 ms —

\frac{1\ \text{hour}}{16.7\ \text{ms}} = \frac{3{,}600{,}000\ \text{ms}}{16.7\ \text{ms}} \approx 216{,}000\times \text{ less time per frame.}

A gap of order 10^5 in the per-frame budget is the reason the two fields evolved different techniques. You do not fight a six-orders-of-magnitude difference with clever coding; you fight it by approximating.

Seeing the gap

Because the two budgets differ by a factor of hundreds of thousands, you can only draw them together on a logarithmic axis — each tick is ten times the last. Even so, the offline bar towers over the real-time sliver.

The convergence

Here is the twist that makes this an exciting moment in graphics: the gap is closing. Several advances are dragging near-offline quality into the real-time budget:

The result is that a modern game can show dynamic global illumination — moving lights bouncing colour around a scene — that would have required a render farm a decade ago.

The gap that remains

Convergence is not equality. Film still spends vastly more per frame to capture the last few percent: thousands of samples for perfectly clean caustics and motion blur, no reprojection artefacts, no upscaling softness, and correctness that holds up when a supervisor scrubs the shot one frame at a time. Real-time buys "good enough at speed"; offline buys "as good as it gets, cost be damned". The techniques are converging, but the priorities — deterministic speed versus uncompromised correctness — still pull them apart.

ML upscalers like DLSS don't guess in a vacuum — they exploit two things. First, they are trained on enormous libraries of ultra-high-resolution frames, so the network has learned what plausible sharp detail looks like (clean edges, fine textures). Second, they use temporal information: the game hands the network motion vectors saying how each pixel moved since the last frame, so detail from previous frames can be reprojected and accumulated into this one. So it isn't hallucinating from nothing — it is fusing many low-res frames plus learned priors into a high-res reconstruction. That is why upscalers can wobble on brand-new detail that has no history (thin wires, particle sparks): there is nothing to accumulate yet.

A natural first thought is: "path tracing looks amazing — let's just run RenderMan inside the game." You can't. The film renderer assumes an hour-scale budget per frame; the game has milliseconds. That is not a 2× or 10× difference you can code around — it is \sim\!10^5. So real-time must approximate: bake static lighting ahead of time, reproject and accumulate across frames, trace only a handful of rays and denoise, render low-res and upscale.

The trap runs the other way too. The real-time tricks that make games fast — baked, static GI, screen-space effects that miss off-screen light — break for film's dynamic, close-scrutiny shots: bake a lightmap and nothing in that light can move; use screen-space reflections and anything off-camera vanishes from the mirror. Under a frame-by-frame director's eye those shortcuts fall apart. Each world's shortcuts are tuned to its budget and its tolerance for error — they don't transplant.