Directional Derivatives

Stand on a hillside with a topographic map in your hand. A partial derivative answers a strangely narrow question: how fast does the height f(x, y) change if you walk due east (f_x) or due north (f_y)? But no trail is that obliging. The path you actually want to take leaves your feet at some bearing of its own — north-north-east, a little south of west, anywhere on the compass. What slope will you feel then?

The directional derivative answers exactly that. Pick a unit direction \mathbf{u} = (u_1, u_2) — your bearing — and measure the rate of change of f as you step off the point along it:

D_{\mathbf{u}} f(a, b) = \lim_{t \to 0} \frac{f(a + t u_1,\; b + t u_2) - f(a, b)}{t}.

The two partials are just the special cases \mathbf{u} = (1, 0) (due east) and \mathbf{u} = (0, 1) (due north). And here is the small miracle this page is about: those two compass readings determine all the others. Once you know the slope east and the slope north, you never need this limit again — every one of the infinitely many bearings has its slope handed to you by a single dot product.

Deriving the formula from the chain rule

Walking along the direction \mathbf{u} traces a straight line. Restricting f to that line turns a two-variable function into an ordinary one-variable function of the step length t — and an ordinary derivative is exactly what the multivariable chain rule knows how to compute.

Step 1 — parametrise the line. Start at (a, b) and move with velocity \mathbf{u}. The position after a step t is

x(t) = a + t u_1, \qquad y(t) = b + t u_2.

Step 2 — define the one-variable restriction. Let g be f seen only along that line:

g(t) = f\big(x(t),\, y(t)\big).

By the very definition above, the directional derivative is the ordinary derivative of g at the start: D_{\mathbf{u}} f(a, b) = g'(0).

Step 3 — differentiate with the chain rule. Since g is f composed with the path (x(t), y(t)), the chain rule gives

g'(t) = f_x\big(x(t), y(t)\big)\, \frac{dx}{dt} + f_y\big(x(t), y(t)\big)\, \frac{dy}{dt}.

Step 4 — read off the velocities. Differentiating the parametrisation in Step 1, the line moves at constant velocity:

\frac{dx}{dt} = u_1, \qquad \frac{dy}{dt} = u_2.

Step 5 — evaluate at the start t = 0, where (x, y) = (a, b):

D_{\mathbf{u}} f(a, b) = g'(0) = f_x(a, b)\, u_1 + f_y(a, b)\, u_2.

Step 6 — recognise a dot product. That sum is exactly the dot product of the vector of partials with the direction. Collecting the partials into the gradient \nabla f = (f_x, f_y),

D_{\mathbf{u}} f = \nabla f \cdot \mathbf{u} = f_x\, u_1 + f_y\, u_2.

No limit, no fuss: two partials and a dot product give the slope in any direction.

Let f be differentiable at (a, b) and let \mathbf{u} = (u_1, u_2) be a unit vector (\|\mathbf{u}\| = 1). Then the directional derivative of f at (a, b) in the direction \mathbf{u} exists and equals D_{\mathbf{u}} f(a, b) = \nabla f(a, b) \cdot \mathbf{u} = f_x(a, b)\, u_1 + f_y(a, b)\, u_2.
In particular D_{(1,0)} f = f_x and D_{(0,1)} f = f_y: the partials are directional derivatives along the axes.

Worked example 1: a clean 3-4-5 bearing

Take f(x, y) = x^2 y at the point (2, 3), walking in the direction \mathbf{u} = \left(\tfrac{3}{5}, \tfrac{4}{5}\right).

Step 1 — the gradient. f_x = 2xy and f_y = x^2, so at (2, 3):

\nabla f(2, 3) = (2 \cdot 2 \cdot 3,\; 2^2) = (12, 4).

Step 2 — check the direction is a unit vector. \left(\tfrac{3}{5}\right)^2 + \left(\tfrac{4}{5}\right)^2 = \tfrac{9}{25} + \tfrac{16}{25} = 1. Yes — the 3-4-5 triangle strikes again. (If it hadn't been, we would normalise first; see the "Watch out!" box below.)

Step 3 — dot them together.

D_{\mathbf{u}} f(2, 3) = (12, 4) \cdot \left(\tfrac{3}{5}, \tfrac{4}{5}\right) = \tfrac{36}{5} + \tfrac{16}{5} = \tfrac{52}{5} = 10.4.

Standing at (2, 3) and stepping off along that bearing, the surface climbs 10.4 units of height per unit of horizontal travel. One number, one slope — that's the whole answer.

Worked example 2: a 45° heading

Now f(x, y) = x^2 + xy at the point (1, 2), heading at 45^\circ from the east axis.

Step 1 — the partials. f_x = 2x + y and f_y = x, so at (1, 2) the gradient is \nabla f(1, 2) = (2\cdot 1 + 2,\; 1) = (4, 1).

Step 2 — a unit direction. Any compass heading \theta gives the unit vector (\cos\theta, \sin\theta) for free — sine and cosine come pre-normalised. Here \mathbf{u} = (\cos 45^\circ, \sin 45^\circ) = \left(\tfrac{1}{\sqrt 2}, \tfrac{1}{\sqrt 2}\right).

Step 3 — dot them together.

D_{\mathbf{u}} f(1, 2) = (4, 1) \cdot \left(\tfrac{1}{\sqrt 2}, \tfrac{1}{\sqrt 2}\right) = \frac{4}{\sqrt 2} + \frac{1}{\sqrt 2} = \frac{5}{\sqrt 2} \approx 3.54.

So f climbs at about 3.54 units of height per unit of horizontal travel in that direction. Hold on to this point and this gradient — (4, 1) at (1, 2) — because the interactive figure below and the next example both live there.

Three traps, in decreasing order of how many exams they have ruined:

\mathbf{u} must be a unit vector — the #1 error. The formula is only a genuine rate when \|\mathbf{u}\| = 1: in Step 1 of the derivation, the walker covers distance \|\mathbf{u}\|\, t as the parameter advances by t, so a non-unit direction measures "rise per parameter", not "rise per metre walked". Scaling the direction scales the answer — \nabla f \cdot (c\mathbf{u}) = c\, (\nabla f \cdot \mathbf{u}) — which is clearly not a property of the terrain. Handed \mathbf{v} = (3, 4)? Feeding it in raw inflates every answer by a factor of \|\mathbf{v}\| = 5. Normalise first: \mathbf{u} = \mathbf{v}/\|\mathbf{v}\| = \left(\tfrac{3}{5}, \tfrac{4}{5}\right).
D_{\mathbf{u}} f is a number, not a vector. It is a slope — like every derivative you have ever met. The vector in the story is the gradient \nabla f; the directional derivative is what you get after dotting it with a direction. If your final answer to "find D_{\mathbf{u}} f" has components, something went wrong.
The formula needs differentiability. The theorem says differentiable \Rightarrow D_{\mathbf{u}} f = \nabla f \cdot \mathbf{u} — not the other way round. There are pathological functions with a perfectly good directional derivative in every direction and yet no gradient formula that works (see the vignette further down). For the polynomials and smooth functions of everyday calculus you are safe, but the fine print is real.

The cosine reading: slope as a dial

The dot product has a geometric face: \mathbf{a} \cdot \mathbf{b} = \|\mathbf{a}\|\, \|\mathbf{b}\| \cos\theta, where \theta is the angle between the vectors. Apply it to \nabla f \cdot \mathbf{u} with \|\mathbf{u}\| = 1 and the whole subject snaps into focus:

D_{\mathbf{u}} f = \|\nabla f\| \cos\theta,

where \theta is now the angle between your bearing and the gradient. Read it like a dial. As you swing your bearing through a full turn, the slope you feel is a pure cosine wave:

\theta = 0 — walking straight along \nabla f: \cos\theta = 1, the slope hits its maximum, +\|\nabla f\|.
\theta = 90^\circ — walking perpendicular to the gradient: \cos\theta = 0, the slope is exactly zero. You are contouring.
\theta = 180^\circ — walking straight against the gradient: \cos\theta = -1, the slope bottoms out at -\|\nabla f\|, the steepest descent.

At our point (1, 2) with \nabla f = (4, 1), the dial reads D_{\mathbf{u}} f = 4\cos\theta + \sin\theta = \sqrt{17}\,\cos(\theta - \theta_0) as the bearing \theta turns (with \theta_0 = \arctan\tfrac14 \approx 14^\circ, the gradient's own bearing). The chart shows one full turn of the dial — a cosine wave pinned between +\sqrt{17} and -\sqrt{17}, crossing zero exactly twice, at the two contouring directions:

Every point of a differentiable surface hides such a dial. The two numbers f_x, f_y set its amplitude and its phase — and that is precisely how two compass readings determine every bearing.

The payoff: which way is steepest — and how steep?

The cosine dial answers the question every hiker, skier and raindrop cares about. Still at (1, 2) on f = x^2 + xy, with \nabla f = (4, 1):

Which direction climbs fastest? The dial peaks at \theta = 0, i.e. when \mathbf{u} points along the gradient itself. As a unit vector:

\mathbf{u}_{\max} = \frac{\nabla f}{\|\nabla f\|} = \frac{(4, 1)}{\sqrt{4^2 + 1^2}} = \left(\tfrac{4}{\sqrt{17}}, \tfrac{1}{\sqrt{17}}\right) \approx (0.970,\; 0.243).

And how steep is that? The peak value of the dial is the gradient's length:

\max_{\|\mathbf{u}\| = 1} D_{\mathbf{u}} f = \|\nabla f\| = \sqrt{17} \approx 4.12.

Sanity check with the raw formula: (4, 1) \cdot \left(\tfrac{4}{\sqrt{17}}, \tfrac{1}{\sqrt{17}}\right) = \tfrac{16 + 1}{\sqrt{17}} = \sqrt{17}. ✓ The same reasoning at the point from Example 1 — \nabla f(2,3) = (12, 4) for f = x^2 y — gives a steepest slope of \|(12, 4)\| = \sqrt{160} = 4\sqrt{10} \approx 12.65, in the unit direction (12, 4)/\sqrt{160} = \left(\tfrac{3}{\sqrt{10}}, \tfrac{1}{\sqrt{10}}\right). Steepest descent is always the exact opposite bearing, -\nabla f / \|\nabla f\|, with slope -\|\nabla f\|.

Where is the slope zero? The dial reads zero at \theta = \pm 90^\circ — the two directions perpendicular to the gradient. For \nabla f = (4, 1), rotate by a quarter turn: \mathbf{u}_0 = \pm\left(\tfrac{-1}{\sqrt{17}}, \tfrac{4}{\sqrt{17}}\right). Check: (4, 1) \cdot \tfrac{1}{\sqrt{17}}(-1, 4) = \tfrac{-4 + 4}{\sqrt{17}} = 0. ✓ Walking that way, your altitude does not change at all — you are tracing the level curve through the point. This is the first glimpse of a beautiful fact you will meet properly with the gradient: the gradient is always perpendicular to the level curves.

Skiers have a name for -\nabla f / \|\nabla f\|: the fall line — the direction a snowball rolls, the path of steepest descent. Point your skis down the fall line and you feel the mountain's full \|\nabla f\|; traverse across it, near the zero of the dial, and even a black run feels gentle. Carving turns is just sweeping \theta back and forth through the cosine dial to keep the felt slope survivable.

Nature and engineers read the same dial. Water runoff follows the fall line (streams run perpendicular to contour lines on every map). Hiking trails do the opposite: switchbacks zig-zag at large \theta so the climb stays near the zero reading. And gradient descent — the algorithm that trains essentially every modern neural network — is directional-derivative shopping: at each step it asks "along which direction does the error f drop fastest?", answers -\nabla f, takes a small step, and repeats — a million-dimensional skier forever seeking the fall line.

Here is why the theorem insists on differentiable. Define

f(x, y) = \frac{x^2 y}{x^4 + y^2} \quad (\text{and } f(0,0) = 0).

At the origin, approach along any straight line \mathbf{u} = (\cos\theta, \sin\theta) and the limit in the definition exists — every single directional derivative is a perfectly good number (it works out to \cos^2\theta / \sin\theta off the x-axis, and 0 along it). Yet the partials at the origin are f_x = f_y = 0, so \nabla f \cdot \mathbf{u} = 0 for every \mathbf{u} — and that contradicts the non-zero slopes we just found. The dot-product formula fails.

The culprit: sneak towards the origin along the parabola y = x^2 and f sits stubbornly at \tfrac{x^2 \cdot x^2}{x^4 + x^4} = \tfrac12 — the function is not even continuous there, let alone differentiable. Straight-line probes miss the misbehaviour hiding between the lines. Moral: having slopes along every line is weaker than being differentiable; only differentiability guarantees a genuine tangent plane and the tidy formula D_{\mathbf{u}} f = \nabla f \cdot \mathbf{u}.

See the slope turn with the direction

The faint rings are level curves of f(x, y) = x^2 + xy — points of equal height, like contour lines on a map. At the marked point P = (1, 2) the gradient is \nabla f = (2x + y,\, x) = (4, 1), drawn as the faint fixed arrow. Swing the unit arrow \mathbf{u} with the slider and watch the live readout of D_{\mathbf{u}} f = \nabla f \cdot \mathbf{u}.

Three experiments to run:

Set \theta = 15^\circ, right on top of the \nabla f arrow — the readout hits its peak, \sqrt{17} \approx 4.12.
Set \theta = 105^\circ, a quarter turn away — the arrow runs along the level ring and the readout collapses to (nearly) zero.
Set \theta = 195^\circ, dead against the gradient — the readout bottoms out at -4.12: the fall line.

One full turn of the slider is one full sweep of the cosine dial from the chart above — peak, zero, trough, zero, and back.

The same slope, on the hill itself

The contour map above flattens the landscape onto the page; here is the hill in three dimensions. A directional derivative is the surface's slope as you walk in a chosen direction — not merely due east (f_x) or due north (f_y), but along any bearing you please. Drag the saddle z = \tfrac12(x^2 - y^2) to spin it: the marked dot sits at a point, and the short segment leaving it shows one such chosen heading. The steepness you feel walking along that segment is D_{\mathbf{u}} f.