Moment-Generating Functions

Suppose you had to describe a random variable to someone — completely. You could hand over its mean. Then its variance. Then \mathbb{E}[X^3] for the skewness, \mathbb{E}[X^4] for the tails, and on and on: an infinite shopping list of moments \mathbb{E}[X^n], each one a separate integral to grind out.

Or you could hand over one function that stores them all. The moment-generating function (MGF) of a random variable X is the expectation

M_X(t) = \mathbb{E}\!\left[e^{tX}\right],

defined for those t where this expectation is finite (we only ever need it in an interval around t = 0). Think of it as a zip file for the distribution: every moment is compressed inside, and the decompression tool is differentiation. Differentiate once and set t = 0 — out pops the mean. Differentiate n times — out pops \mathbb{E}[X^n]. The dummy variable t is just the crank handle; it means nothing by itself.

And the MGF has a second superpower that no list of moments offers: it turns the hardest everyday operation in probability — adding independent random variables, which for densities means an ugly convolution integral — into ordinary multiplication of functions. Those two tricks together are why the MGF sits behind so much of quantitative finance: pricing formulas, the behaviour of summed returns, and the slickest known proof of the central limit theorem all run on it.

How it generates moments, line by line

The claim is that the n-th derivative of M_X at the origin returns the n-th moment \mathbb{E}[X^n]. The proof is a power series carried through the expectation. Start from the exponential series — the Taylor series of e^{u} with u = tX:

e^{tX} = \sum_{n=0}^{\infty} \frac{(tX)^n}{n!} = 1 + tX + \frac{t^2}{2!}X^2 + \frac{t^3}{3!}X^3 + \cdots

Step 1 — take expectations term by term. Expectation is linear, so it passes through the sum (the interchange of \mathbb{E} and \sum is justified when the MGF is finite near 0). The factors t^n/n! are constants and come outside \mathbb{E}, leaving the moments:

M_X(t) = \mathbb{E}\!\left[\sum_{n=0}^{\infty} \frac{t^n}{n!}X^n\right] = \sum_{n=0}^{\infty} \frac{t^n}{n!}\,\mathbb{E}[X^n].

So M_X is a power series in t whose coefficients are the moments (scaled by 1/n!) — the zip file laid out byte by byte. Now we read them off by evaluating and differentiating at t = 0.

Step 2 — evaluate at t = 0. Every term with n \ge 1 carries a factor t^n and vanishes, leaving only the n = 0 term \mathbb{E}[X^0] = \mathbb{E}[1] = 1:

M_X(0) = \sum_{n=0}^{\infty} \frac{0^n}{n!}\,\mathbb{E}[X^n] = 1.

Step 3 — differentiate once. Differentiate the series term by term; \frac{d}{dt}\,t^n = n\,t^{n-1}, and the n = 0 constant term drops out:

M_X'(t) = \sum_{n=1}^{\infty} \frac{n\,t^{n-1}}{n!}\,\mathbb{E}[X^n] = \sum_{n=1}^{\infty} \frac{t^{n-1}}{(n-1)!}\,\mathbb{E}[X^n].

Set t = 0: again every term with a positive power of t dies, and only the n = 1 term survives, with t^{0}/0! = 1:

M_X'(0) = \mathbb{E}[X].

Step 4 — differentiate again. One more derivative pulls down another factor:

M_X''(t) = \sum_{n=2}^{\infty} \frac{t^{n-2}}{(n-2)!}\,\mathbb{E}[X^n], \qquad M_X''(0) = \mathbb{E}[X^2].

Step 5 — the general pattern. Differentiating n times turns t^n/n! into the constant 1 and kills every lower term, while every higher term still carries a positive power of t that vanishes at the origin. Only the n-th term survives:

M_X^{(n)}(0) = \mathbb{E}[X^n].

So every moment is the value of a derivative at the origin — and in particular the variance is always the two-derivative combination \operatorname{Var}(X) = \mathbb{E}[X^2] - \mathbb{E}[X]^2 = M_X''(0) - \bigl(M_X'(0)\bigr)^2.

Let M_X(t) = \mathbb{E}[e^{tX}] be finite on an interval around t = 0. Then for every n \ge 0, M_X^{(n)}(0) = \mathbb{E}[X^n]; in particular M_X(0) = 1, M_X'(0) = \mathbb{E}[X], and M_X''(0) = \mathbb{E}[X^2]. Moreover, for independent X and Y, M_{X+Y}(t) = M_X(t)\,M_Y(t).

Worked example: the humble coin, from scratch

Nothing builds trust in machinery like running it on the simplest possible input. Let X \sim \text{Bernoulli}(p): a biased coin that shows X = 1 with probability p and X = 0 with probability 1 - p.

Step 1 — compute the MGF from the definition. For a discrete variable the expectation is a weighted sum over the outcomes:

M_X(t) = \mathbb{E}\!\left[e^{tX}\right] = (1-p)\,e^{t\cdot 0} + p\,e^{t\cdot 1} = 1 - p + p\,e^{t}.

Step 2 — sanity check at the origin. M_X(0) = 1 - p + p = 1. ✓ (Every legitimate MGF passes through 1 at t = 0 — the probabilities must sum to one. It's the first thing to check in any computation.)

Step 3 — differentiate for the mean. The constant 1 - p dies; the exponential survives:

M_X'(t) = p\,e^{t} \quad\Rightarrow\quad M_X'(0) = p = \mathbb{E}[X]. \checkmark

Step 4 — differentiate again for the second moment. M_X''(t) = p\,e^{t} too, so M_X''(0) = p = \mathbb{E}[X^2]. That looks odd until you notice that X only takes the values 0 and 1, so X^2 = X — the machinery quietly knew that.

Step 5 — assemble the variance.

\operatorname{Var}(X) = M_X''(0) - \bigl(M_X'(0)\bigr)^2 = p - p^2 = p(1-p),

the classic Bernoulli variance, maximal for a fair coin (p = \tfrac12 gives \tfrac14). Every answer came out of one function by turning the crank.

Now watch the product rule earn its keep. Flip the coin n times independently and let S = X_1 + \cdots + X_n be the number of heads. Sums of independent variables multiply their MGFs, so

M_S(t) = \bigl(1 - p + p\,e^{t}\bigr)^{n}.

Expand by the binomial theorem and you can literally read the binomial distribution off the coefficients of e^{tk}: M_S(t) = \sum_{k=0}^{n} \binom{n}{k} p^k (1-p)^{n-k} e^{tk} — each coefficient is \mathbb{P}(S = k). One line of algebra replaced an n-fold convolution.

The multiplication superpower — and a small toolkit

Sums of independent variables multiply. This is the property that makes the MGF the natural tool for adding random variables. Start from the definition at X + Y and split the exponential of a sum into a product of exponentials:

M_{X+Y}(t) = \mathbb{E}\!\left[e^{t(X+Y)}\right] = \mathbb{E}\!\left[e^{tX}\,e^{tY}\right].

When X and Y are independent, so are e^{tX} and e^{tY} (functions of independent variables), and the expectation of a product of independent things is the product of their expectations:

\mathbb{E}\!\left[e^{tX}\,e^{tY}\right] = \mathbb{E}\!\left[e^{tX}\right]\mathbb{E}\!\left[e^{tY}\right] = M_X(t)\,M_Y(t).

So adding independent variables turns into multiplying their MGFs — a sum in the sample space becomes a product in MGF-land. If you know Fourier or Laplace transforms, this should feel familiar: convolution on one side, multiplication on the other. It is exactly why MGFs make sums of normals so easy to handle, as we'll see below.

Two more rules complete the everyday toolkit, both one-line substitutions into the definition. Scaling: for a constant a,

M_{aX}(t) = \mathbb{E}\!\left[e^{t(aX)}\right] = \mathbb{E}\!\left[e^{(at)X}\right] = M_X(at)

— stretching the variable compresses or stretches the argument of the MGF. Shifting: for a constant c,

M_{X+c}(t) = \mathbb{E}\!\left[e^{t(X+c)}\right] = e^{ct}\,\mathbb{E}\!\left[e^{tX}\right] = e^{ct}\,M_X(t),

since e^{ct} is deterministic and slides out of the expectation. With multiply-for-sums, scale, and shift, you can build the MGF of any affine combination of independent variables without touching an integral.

When M_X(t) is finite on an open interval containing 0, it pins down the distribution uniquely: two random variables with the same MGF (finite near 0) have the same law. This is what lets us recognise a distribution by computing its MGF and matching it to a known form — the move we lean on constantly below.

The reason runs deeper than the moments. Knowing all the moments \mathbb{E}[X^n] is, in general, not enough to fix a distribution — there are genuinely different distributions sharing every moment (the lognormal is the classic culprit). What the MGF adds is analyticity: if the series \sum_n \mathbb{E}[X^n]\,t^n/n! converges in a neighbourhood of 0, then M_X is an analytic function there. Extending t to imaginary values t = is turns the MGF into the characteristic function \mathbb{E}[e^{isX}] — a Fourier transform of the distribution — and a Fourier transform can be inverted to recover the distribution. The finiteness-near-zero condition is precisely what guarantees this analytic continuation exists. Without it (as for the lognormal, whose MGF is infinite for every t > 0), the moments alone leave room for impostors.

The normal MGF — a quadratic wearing an exponential

For X \sim N(\mu, \sigma^2) the integral works out to a clean exponential of a quadratic in t:

M_X(t) = \exp\!\left(\mu t + \tfrac12\sigma^2 t^2\right).

Learn to read this formula, because you will meet it constantly: the coefficient of t in the exponent is the mean, and twice the coefficient of t^2 is the variance. Check it with the crank: M'(t) = (\mu + \sigma^2 t)\,M(t), so M'(0) = \mu; differentiating again and setting t=0 gives M''(0) = \mu^2 + \sigma^2 = \mathbb{E}[X^2], so the variance \mathbb{E}[X^2] - \mu^2 = \sigma^2 drops out as it should.

The curve below is the whole story made visible. Slide \mu and \sigma and watch three things: the curve always passes through M(0) = 1 (Step 2 above — no slider can move that point); its slope at t = 0 is \mu (drag the mean negative and the curve tips downhill through the anchor); and its curvature at the origin grows with \sigma — a more spread-out distribution makes e^{tX} blow up faster in both directions. An MGF is always convex (its second derivative is \mathbb{E}[X^2 e^{tX}] \ge 0), and the normal is the extreme case of tidiness: take logarithms and everything above the quadratic vanishes.

Where does \exp(\mu t + \tfrac12\sigma^2 t^2) come from? It is an expectation of e^{tX} against the N(\mu,\sigma^2) density — the same completing-the-square manoeuvre as the lognormal mean, but kept general in t:

M_X(t) = \mathbb{E}\!\left[e^{tX}\right] = \int_{-\infty}^{\infty} e^{tx}\,\frac{1}{\sigma\sqrt{2\pi}}\,\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)dx.

Merge the two exponentials and put the exponent over the common denominator 2\sigma^2, expanding (x-\mu)^2 = x^2 - 2\mu x + \mu^2:

tx - \frac{(x-\mu)^2}{2\sigma^2} = \frac{2\sigma^2 t x - (x^2 - 2\mu x + \mu^2)}{2\sigma^2} = \frac{-x^2 + 2(\mu + \sigma^2 t)x - \mu^2}{2\sigma^2}.

Complete the square in the numerator, aiming for -(x - a)^2 + (\text{const}) with a = \mu + \sigma^2 t. Since -(x - a)^2 = -x^2 + 2ax - a^2 matches the first two terms, the numerator equals

-\bigl(x - (\mu + \sigma^2 t)\bigr)^2 + (\mu + \sigma^2 t)^2 - \mu^2.

Expand the leftover constant: (\mu + \sigma^2 t)^2 - \mu^2 = \mu^2 + 2\mu\sigma^2 t + \sigma^4 t^2 - \mu^2 = 2\mu\sigma^2 t + \sigma^4 t^2. Dividing by 2\sigma^2 gives the constant that escapes the integral:

\frac{2\mu\sigma^2 t + \sigma^4 t^2}{2\sigma^2} = \mu t + \tfrac12 \sigma^2 t^2.

The exponent is now a constant plus a perfect-square Gaussian term:

M_X(t) = e^{\mu t + \frac12 \sigma^2 t^2}\cdot \frac{1}{\sigma\sqrt{2\pi}}\int_{-\infty}^{\infty} \exp\!\left(-\frac{\bigl(x - (\mu + \sigma^2 t)\bigr)^2}{2\sigma^2}\right)dx.

The remaining integral is the total area under an N(\mu + \sigma^2 t,\, \sigma^2) density — a normal merely shifted to a new centre — so it equals 1, and only the front constant survives:

M_X(t) = \exp\!\left(\mu t + \tfrac12 \sigma^2 t^2\right).

Adding normals without a single integral

Here is the payoff of the whole framework. Let X \sim N(\mu_1, \sigma_1^2) and Y \sim N(\mu_2, \sigma_2^2) be independent. What is the distribution of X + Y? Done directly, that's a convolution of two Gaussian densities — a page of completing the square. Done with MGFs, it's one line: multiply, and exponents add.

M_{X+Y}(t) = M_X(t)\,M_Y(t) = e^{\mu_1 t + \frac12\sigma_1^2 t^2}\, e^{\mu_2 t + \frac12\sigma_2^2 t^2} = \exp\!\Bigl((\mu_1 + \mu_2)\,t + \tfrac12\bigl(\sigma_1^2 + \sigma_2^2\bigr)\,t^2\Bigr).

Now read the result the way we learned above: this is exactly the MGF of a normal with mean \mu_1 + \mu_2 and variance \sigma_1^2 + \sigma_2^2. And because a finite MGF determines the distribution (the vignette above), we may conclude — not just guess — that

X + Y \sim N\!\bigl(\mu_1 + \mu_2,\; \sigma_1^2 + \sigma_2^2\bigr).

Means add, variances add, and the sum stays normal. Note what adds: the variances \sigma^2, never the standard deviations \sigma.

This one line is a workhorse of finance. If a stock's daily log-returns are independent N(\mu, \sigma^2), then the log-return over n days is their sum — so it is N(n\mu,\, n\sigma^2), with standard deviation \sigma\sqrt{n}. That is the famous square-root-of-time rule for volatility ("annualise a daily vol by \sqrt{252}"), and it falls straight out of multiplying MGFs.

The lognormal connection: a formula you already know

You met the mysterious-looking mean of the lognormal distribution: if S = e^{Z} with Z \sim N(\mu, \sigma^2), then \mathbb{E}[S] = e^{\mu + \sigma^2/2}, with that curious extra \sigma^2/2. Look at it again with fresh eyes:

\mathbb{E}[S] = \mathbb{E}\!\left[e^{Z}\right] = \mathbb{E}\!\left[e^{1 \cdot Z}\right] = M_Z(1) = \exp\!\left(\mu \cdot 1 + \tfrac12\sigma^2 \cdot 1^2\right) = e^{\mu + \sigma^2/2}.

The lognormal mean is the normal MGF evaluated at t = 1. The \sigma^2/2 was never mysterious — it's the quadratic term of the MGF, the mathematical footprint of convexity: big upward swings of Z inflate e^{Z} more than equal downward swings deflate it. Better still, the same move hands you every moment of the lognormal at once:

\mathbb{E}[S^n] = \mathbb{E}\!\left[e^{nZ}\right] = M_Z(n) = e^{n\mu + n^2\sigma^2/2}.

Mean, mean-square, variance of a lognormal asset price — all just the normal MGF read at t = 1, 2, \dots But notice how violently e^{n^2\sigma^2/2} grows with n. That super-exponential growth of the moments is a warning flag, and it leads directly to the trap below.

Three classic traps, all live ammunition in finance:

If X has density f, then M_X(-s) = \int e^{-sx} f(x)\,dx — which is exactly the Laplace transform of f, the same machine electrical engineers crank on circuits and control systems. The MGF isn't a probabilist's invention at all; it's transform theory wearing a probability hat. And the payoff is the same in both worlds: a hard operation (convolution — adding independent variables, or feeding a signal through a filter) becomes easy multiplication on the transform side, and a transform determines what it came from. Engineers keep tables of Laplace transforms; probabilists keep tables of MGFs. Same tables, different notation.

The most beautiful dividend is the slickest known proof of the central limit theorem. Take iid standardized variables Z_i (mean 0, variance 1, MGF finite near 0) and set S_n = (Z_1 + \cdots + Z_n)/\sqrt{n}. Then:

M_{S_n}(t) = \Bigl[M_Z\!\bigl(t/\sqrt{n}\bigr)\Bigr]^n = \Bigl[1 + \frac{t^2}{2n} + o\!\bigl(\tfrac1n\bigr)\Bigr]^n \;\longrightarrow\; e^{t^2/2}.

Line one: independence turns the sum into a power, and scaling turns t into t/\sqrt{n}. Line two: Taylor's theorem — the MGF near zero is 1 + \mathbb{E}[Z]\,u + \tfrac12\mathbb{E}[Z^2]u^2 + \cdots, and the mean-zero, variance-one assumptions kill the linear term and fix the quadratic. Line three: the compound-interest limit (1 + x/n)^n \to e^{x}. The limit e^{t^2/2} is the MGF of N(0,1) — and since MGFs determine distributions, S_n converges to the standard normal. The bell curve's universality, in three lines of algebra. (The fully general proof swaps in characteristic functions, for the fat-tailed cases above — but the idea is identical.)