σ-Algebras

Before we can talk about the probability of an event, we have to be careful about which collections of outcomes even count as events. Fix a sample space \Omega — the set of all possible outcomes. An event is a subset A \subseteq \Omega. A σ-algebra \mathcal{F} is a collection of such subsets, chosen so that the natural set operations never lead us outside the collection.

Concretely, \mathcal{F} is a σ-algebra on \Omega when it satisfies three axioms:

The whole space is an event: \Omega \in \mathcal{F}.
Closed under complement: if A \in \mathcal{F} then A^{c} \in \mathcal{F}.
Closed under countable unions: if A_1, A_2, \dots \in \mathcal{F} then \bigcup_{i=1}^{\infty} A_i \in \mathcal{F}.

The algebra part is old terminology: an algebra of sets is a collection closed under complement and finite unions and intersections — the set operations behave like an algebra, much as + and \times do for numbers.

The \sigma (lower-case Greek sigma) is the analyst's shorthand for “countable” — it traces back to the German Summe (“sum”), as in a countable sum. Adding the σ promotes that closure rule from finite to countable: a σ-algebra is an algebra of sets that is also closed under countable unions \bigcup_{i=1}^{\infty} A_i. The same σ shows up again in σ-finite measures and in F_{\sigma} sets — everywhere it quietly means “countably many”.

What the axioms give you for free

Only three rules were assumed — yet they quietly hand you several more. Let us extract them one careful step at a time, never using anything but the three axioms.

1. The empty set is always an event. The empty set \emptyset is the complement of the whole space, because no outcome lies outside \Omega:

\emptyset = \Omega^{c}.

By the first axiom \Omega \in \mathcal{F}, and by the second axiom (closure under complement) the complement of any member is again a member. Applying that to \Omega:

\Omega \in \mathcal{F} \;\Longrightarrow\; \Omega^{c} \in \mathcal{F} \;\Longrightarrow\; \emptyset \in \mathcal{F}.

2. Closure under countable intersections. The axioms mention unions but say nothing about intersections — yet intersections come for free, via de Morgan's law. Start with a countable family A_1, A_2, \dots \in \mathcal{F}. De Morgan rewrites their intersection as the complement of a union of complements:

\bigcap_{i=1}^{\infty} A_i = \left(\bigcup_{i=1}^{\infty} A_i^{c}\right)^{c}.

Now read the right-hand side from the inside out, checking membership at each stage:

Each A_i \in \mathcal{F}, so by closure under complement every A_i^{c} \in \mathcal{F}.
The A_i^{c} are countably many members of \mathcal{F}, so by closure under countable unions \bigcup_{i} A_i^{c} \in \mathcal{F}.
That union is itself a member, so by closure under complement its complement \left(\bigcup_{i} A_i^{c}\right)^{c} \in \mathcal{F} too.

But that complement is exactly \bigcap_{i} A_i, so the intersection is an event. Nothing beyond the three axioms was used.

The point of all this is consistency: if "rain" and "wind" are events we can measure, then "not rain", "rain or wind", and "rain and wind" had better be measurable as well. A σ-algebra is exactly the smallest amount of structure that makes the algebra of events closed under everything we want to do with them — a prerequisite drawn from set theory.

Let \mathcal{F} be a σ-algebra on \Omega. Then the empty set is an event, \emptyset \in \mathcal{F}, and \mathcal{F} is closed under countable intersections: if A_1, A_2, \dots \in \mathcal{F} then \bigcap_{i=1}^{\infty} A_i \in \mathcal{F}. (Both follow from the three axioms alone — together with finite unions and intersections, obtained by padding a finite family with copies of \emptyset.)

See closure in action

Take \Omega = \{1,2,3,4,5,6\}, the faces of a die, and two events A=\{2,4,6\} and B=\{4,5\}. Step through the cascade: each event drags in its complement, every union drags in its complement too (so, by de Morgan, the intersections as well) — and it doesn't stop. The collection keeps growing until it contains every set the operations can produce: the σ-algebra generated by A and B.

Three σ-algebras worth knowing

The trivial σ-algebra \{\emptyset, \Omega\} — the smallest possible, knowing only "nothing" and "everything".
The power set 2^{\Omega} — the largest, containing every subset. Perfect for a finite or countable \Omega.
The generated σ-algebra \sigma(\mathcal{A}) — the smallest σ-algebra containing a given collection \mathcal{A}. On \mathbb{R} the most important example is the Borel σ-algebra \mathcal{B}(\mathbb{R}), generated by the open sets — it is the home of every event we will ever ask about a real-valued measurement.

What is the smallest σ-algebra \sigma(A) that contains a single event A \subseteq \Omega? Start with only A and force in whatever the three axioms demand, one step at a time — adding nothing that is not required.

We must have A itself.
By the first axiom, the whole space is forced in: \Omega \in \sigma(A).
By closure under complement applied to A, its complement is forced in: A^{c} \in \sigma(A).
By closure under complement applied to \Omega, the empty set is forced in: \emptyset = \Omega^{c} \in \sigma(A).

So far we are obliged to include the four sets

\{\,\emptyset,\; A,\; A^{c},\; \Omega\,\}.

Now check that this collection is already closed — that the axioms cannot force in a fifth set. Complementing each member stays inside the family,

\emptyset^{c} = \Omega, \qquad \Omega^{c} = \emptyset, \qquad A^{c} \text{ and } (A^{c})^{c} = A,

and every union of members lands back inside it as well — for instance A \cup A^{c} = \Omega, A \cup \emptyset = A, and A \cup \Omega = \Omega (intersections follow by de Morgan). Since no operation escapes the four sets, the collection satisfies all three axioms, and nothing smaller could contain A. Therefore

\sigma(A) = \{\,\emptyset,\; A,\; A^{c},\; \Omega\,\}.

(The lone degenerate case is A = \emptyset or A = \Omega, where the four sets collapse to the trivial σ-algebra \{\emptyset, \Omega\}.)

Once you know the recipe, σ-algebras turn up everywhere. A few more worth meeting:

Generated by one event. The smallest σ-algebra containing a single event A \subseteq \Omega is just \{\emptyset, A, A^{c}, \Omega\} — four sets, no more.
Generated by a random variable. For a random variable X, the collection \sigma(X) = \{\,X^{-1}(B) : B \in \mathcal{B}(\mathbb{R})\,\} holds every event you could decide just by observing X — it is exactly the information that X carries.
The product σ-algebra. On a product space \Omega_1 \times \Omega_2, the σ-algebra \mathcal{F}_1 \otimes \mathcal{F}_2 is generated by the rectangles A \times B — the natural home of joint behaviour.
Borel in higher dimensions. \mathcal{B}(\mathbb{R}^{n}), generated by the open sets of \mathbb{R}^{n}, lifts the real-line Borel sets to several dimensions at once.
The Lebesgue σ-algebra. Complete \mathcal{B}(\mathbb{R}) by also admitting every subset of a measure-zero set; the result is strictly larger than the Borel σ-algebra and is where Lebesgue measure most comfortably lives.