Moment-Generating Functions

Every random variable carries a little family of summary numbers called its moments: the mean E(X), then E(X^2), then E(X^3), and so on forever. The first two build the expected value and the variance; the higher ones control skewness, fat tails, and finer shape. Chasing them one at a time is tedious.

The moment-generating function (MGF) is a single, magical function that packs all of those numbers into one object. You feed it a dummy variable t, and it hands you a function whose derivatives at the origin spit the moments back out, one after another. Even better, that one function turns out to be a complete fingerprint of the distribution — know the MGF and you know the whole thing.

Why it "generates" the moments

The name is a promise, and it is kept by a Taylor series. Expand the exponential inside the expectation:

e^{tX} = 1 + tX + \frac{(tX)^2}{2!} + \frac{(tX)^3}{3!} + \cdots

Now take the expectation of both sides. Because expectation is linear, it slides through the sum, and each t^n comes out front:

M_X(t) = 1 + E(X)\,t + \frac{E(X^2)}{2!}\,t^2 + \frac{E(X^3)}{3!}\,t^3 + \cdots

Read that carefully: the coefficient of t^n is exactly E(X^n)/n!. So the n-th moment is hiding as the n-th Taylor coefficient — and the way to pull out a Taylor coefficient is to differentiate n times and evaluate at t = 0:

Differentiating M_X(t) = 1 + E(X)t + \tfrac{E(X^2)}{2}t^2 + \cdots once gives E(X) + E(X^2)t + \cdots, which at t=0 leaves just E(X); every higher term still carries a factor of t and vanishes. That is the whole trick.

Worked example 1 — Bernoulli

A Bernoulli variable is a single coin-flip trial: X = 1 with probability p (a "success") and X = 0 with probability 1-p. Its MGF is a sum over just those two outcomes:

M_X(t) = e^{t \cdot 0}(1-p) + e^{t \cdot 1}\,p = (1-p) + p\,e^{t}.

Now generate the moments. Differentiate with respect to t (note 1-p is a constant, so it dies):

M_X'(t) = p\,e^{t}, \qquad M_X''(t) = p\,e^{t}.

Evaluate each at t = 0, where e^0 = 1:

See the mean as a slope

Because M_X'(0) = E(X), the mean is literally the slope of the MGF at the origin. Below is the Bernoulli MGF M_X(t) = (1-p) + p\,e^{t} together with its tangent line at t = 0. Every curve passes through the same point (0, 1) (since M_X(0) = 1), and the steepness of the tangent there is the mean p. Slide p and watch the tangent tilt: more success probability, steeper slope, larger mean.

Worked example 2 — Poisson

A Poisson variable counts rare events (calls to a helpdesk in an hour, typos on a page) with a rate \lambda. Its MGF is a classic:

M_X(t) = e^{\lambda\left(e^{t} - 1\right)}.

Differentiate with the chain rule — the exponent \lambda(e^{t}-1) has derivative \lambda e^{t}:

M_X'(t) = \lambda e^{t}\,e^{\lambda\left(e^{t}-1\right)}.

At t = 0 the outer exponential is e^{\lambda(1-1)} = e^0 = 1 and e^0 = 1, so

E(X) = M_X'(0) = \lambda \cdot 1 \cdot 1 = \lambda.

The mean of a Poisson is its rate \lambda. Differentiate once more (product + chain rule) and evaluate at 0 to get E(X^2) = \lambda^2 + \lambda, hence \operatorname{Var}(X) = (\lambda^2 + \lambda) - \lambda^2 = \lambda — the Poisson's famous "mean equals variance", both equal to \lambda, extracted from one tidy function.

A table of MGFs

You rarely re-derive these from scratch — you look them up, then differentiate. Here are four workhorses. Notice how each MGF is built from the distribution's own parameters, and how M(0) = 1 for every row.

Distribution M_X(t) = E(e^{tX}) Mean = M'(0)
Bernoulli(p)(1-p) + p\,e^{t}p
Binomial(n, p)\big((1-p) + p\,e^{t}\big)^{n}np
Poisson(\lambda)e^{\lambda(e^{t}-1)}\lambda
Normal N(\mu, \sigma^2)\exp\!\big(\mu t + \tfrac{1}{2}\sigma^2 t^2\big)\mu

Look at the Binomial row next to the Bernoulli row: the Binomial MGF is the Bernoulli MGF raised to the n-th power. That is not a coincidence — it is the second superpower of MGFs, coming right up.

Two superpowers: fingerprint and multiplication

Beyond generating moments, the MGF earns its keep through two structural properties that make hard probability problems easy.

Why does the product rule hold? Independence lets an expectation of a product split into a product of expectations: M_{X+Y}(t) = E\!\left(e^{t(X+Y)}\right) = E\!\left(e^{tX} e^{tY}\right) = E\!\left(e^{tX}\right) E\!\left(e^{tY}\right) = M_X(t) M_Y(t).

Put the two superpowers together and you get slick proofs. Add n independent Bernoulli(p) variables: the product of their MGFs is \big((1-p)+p\,e^{t}\big)^{n} — which is the Binomial MGF from the table. By uniqueness, the sum is Binomial(n,p). The same move shows that a sum of independent normals is normal: multiply \exp(\mu_1 t + \tfrac12\sigma_1^2 t^2) by \exp(\mu_2 t + \tfrac12\sigma_2^2 t^2), add the exponents, and you get the MGF of N(\mu_1+\mu_2,\ \sigma_1^2+\sigma_2^2). No integral in sight.

It really is like DNA. The Taylor coefficients of M_X(t) are the moments — the mean, the spread, the skew, the tail weight, all of it — so the single function encodes every quantitative feature of the shape at once. And uniqueness says no two different distributions share it. That is why probabilists reach for the MGF (and its close cousin the characteristic function E(e^{itX}), which always exists) to prove that some limiting distribution is the normal — a slick route to the central limit theorem: show the MGFs converge, and the distributions must follow.