σ-Algebras

Before we can talk about the probability of an event, we have to be careful about which collections of outcomes even count as events. Fix a sample space \Omega — the set of all possible outcomes. An event is a subset A \subseteq \Omega. A σ-algebra \mathcal{F} is a collection of such subsets, chosen so that the natural set operations never lead us outside the collection.

Concretely, \mathcal{F} is a σ-algebra on \Omega when it satisfies three axioms:

The whole space is an event: \Omega \in \mathcal{F}.
Closed under complement: if A \in \mathcal{F} then A^{c} \in \mathcal{F}.
Closed under countable unions: if A_1, A_2, \dots \in \mathcal{F} then \bigcup_{i=1}^{\infty} A_i \in \mathcal{F}.

The algebra part is old terminology: an algebra of sets is a collection closed under complement and finite unions and intersections — the set operations behave like an algebra, much as + and \times do for numbers.

The \sigma (lower-case Greek sigma) is the analyst's shorthand for “countable” — it traces back to the German Summe (“sum”), as in a countable sum. Adding the σ promotes that closure rule from finite to countable: a σ-algebra is an algebra of sets that is also closed under countable unions \bigcup_{i=1}^{\infty} A_i. The same σ shows up again in σ-finite measures and in F_{\sigma} sets — everywhere it quietly means “countably many”.

What the axioms give you for free

These three rules quietly hand you several more. Since \Omega \in \mathcal{F} and the collection is closed under complement, the empty set is always an event: \emptyset = \Omega^{c} \in \mathcal{F}. And because \bigcap_i A_i = \left(\bigcup_i A_i^{c}\right)^{c} (de Morgan), complement plus countable unions force closure under countable intersections too.

The point of all this is consistency: if "rain" and "wind" are events we can measure, then "not rain", "rain or wind", and "rain and wind" had better be measurable as well. A σ-algebra is exactly the smallest amount of structure that makes the algebra of events closed under everything we want to do with them — a prerequisite drawn from set theory.

See closure in action

Take \Omega = \{1,2,3,4,5,6\}, the faces of a die. Step through: an event A=\{2,4,6\} drags in its complement, and a second event B drags in the union. To be a σ-algebra the collection must contain everything these operations produce.

Three σ-algebras worth knowing

The trivial σ-algebra \{\emptyset, \Omega\} — the smallest possible, knowing only "nothing" and "everything".
The power set 2^{\Omega} — the largest, containing every subset. Perfect for a finite or countable \Omega.
The generated σ-algebra \sigma(\mathcal{A}) — the smallest σ-algebra containing a given collection \mathcal{A}. On \mathbb{R} the most important example is the Borel σ-algebra \mathcal{B}(\mathbb{R}), generated by the open sets — it is the home of every event we will ever ask about a real-valued measurement.