Adversary Arguments
Most of complexity analysis is about a particular algorithm: you write it down, count its
steps, and report its cost. A lower bound is a wilder, more ambitious claim. It says
nothing about any one program — it says that every conceivable algorithm that solves
this problem, including ones nobody has invented yet, must do at least this much work. It is a
statement about the problem itself, a law of nature for computation: no cleverness will ever get below
this floor.
How could you possibly prove something about algorithms you have never seen? You can't run them. The
trick is to argue from the other side of the table. Imagine an adversary who
controls the input and answers the algorithm's questions as evasively as the rules allow — always
keeping the true answer as uncertain as possible. If the adversary can force any algorithm to
ask at least k questions before the answer is pinned down, then
k is a lower bound. This page builds that technique from the ground up.
The adversary: an input that isn't fixed yet
Normally we picture the input as fixed before the algorithm starts. The adversary argument turns this
on its head. The adversary does not commit to an input in advance. Instead it answers
each query — "is A[i] < A[j]?", "what is A[i]?"
— adaptively, choosing the reply that keeps the largest number of possible inputs
still consistent with everything said so far. It is lazy in the most useful way: it only ever pins
down as much of the input as the algorithm has forced it to.
The single rule the adversary must obey is consistency: at the end there must exist at
least one genuine input agreeing with every answer it gave. As long as two different correct
outputs are still possible, the algorithm cannot have finished — if it stopped and announced an
answer, the adversary would produce a consistent input for which that answer is wrong. So the
algorithm is trapped: it must keep asking until only one output survives, and the adversary's whole job
is to make that take as long as possible.
- Answer every query to keep the set of consistent inputs (and hence the set of still-possible
answers) as large as you can.
- The algorithm may only halt once a single answer is forced.
- The number of queries you can drag it through, in the worst case, is a
lower bound for the problem.
Worked example 1 — finding the maximum needs n-1 comparisons
Suppose the only operation allowed is comparing two elements. To find the largest of
n items, the obvious scan does n-1 comparisons.
Can any algorithm do better? No — and here is the clean argument, an accounting flavour of the
adversary idea. Think of the numbers as players in a knockout tournament: each comparison is one match.
Call an element a loser once it has been on the smaller side of some comparison. The
maximum is exactly the one element that never loses. Every comparison produces at most
one new loser. To be certain which element is the max, all
n-1 of the other elements must have lost at least once — otherwise two
unbeaten elements remain and the adversary can still make either one the maximum. Producing
n-1 losers needs at least n-1 comparisons.
The floor and the ceiling meet: finding the maximum is exactly
n - 1 \text{ comparisons} \quad(\Theta(n)).
Worked example 2 — max and min in \lceil 3n/2 \rceil - 2
Now find the largest and the smallest together. The naive plan — find the max in
n-1, then the min of the rest in n-2 — costs
2n-3. Surprisingly you can do better, and the adversary tells you exactly how
much better. The clever algorithm processes elements in pairs: compare the two of a
pair to each other first (1 comparison), then the larger against the running max and the smaller
against the running min (2 more). That is 3 comparisons for every 2 elements, giving
\left\lceil \frac{3n}{2} \right\rceil - 2 \text{ comparisons.}
The matching lower bound is a weighting argument. Give each element two "labels": it could still be the
max, and it could still be the min. At the start there are 2n labels of
uncertainty; at the end only 2 remain (the true max keeps its max-label, the
true min keeps its min-label). The adversary answers so that a comparison between two
never-compared elements removes at most 2 labels, while any other comparison
removes at most 1. Grinding 2n-2 labels away under those
limits forces at least \lceil 3n/2 \rceil - 2 comparisons — matching the
algorithm exactly.
Worked example 3 — searching an unsorted list needs n probes
You want to know whether a value x appears in an unsorted array of
n elements. Each probe reveals one entry. The adversary strategy is almost
insultingly simple: whenever the algorithm probes a position it has not seen, the adversary answers
with a value different from x. As long as even one position
remains unprobed, the adversary is free to declare that unseen slot equals x
(answer "present") or leave it different (answer "absent") — both outputs are still consistent.
So an algorithm that probes fewer than n positions and announces "absent" can
be refuted by putting x in the slot it skipped; one that announces "present"
can be refuted by making every unseen slot different. The algorithm must inspect all
n entries. Unstructured search is \Theta(n),
and this is exactly why sorting first (or hashing) pays off: it buys the structure that lets
binary search beat this floor.
There are two great families of lower-bound argument and they feel different. An
information-theoretic bound counts outcomes: if an algorithm must distinguish
N possible answers and each query has only b
possible replies, it needs at least \log_b N queries, because that many
yes/no bits are required just to name the answer. That is how the
decision-tree
bound for sorting works — n! possible orderings force
\log_2(n!) = \Theta(n\log n) comparisons.
An adversary bound is more dynamic: it doesn't just count outcomes, it plays
against the algorithm's actual sequence of queries, exploiting the fact that each answer only rules out
a little. The two often prove the same theorem, but the adversary shines when a raw outcome count is
too weak — finding the max has only "which of n" outcomes, an
information-theoretic \log_2 n, yet the adversary's per-comparison accounting
gives the true, much larger n-1. When counting outcomes underdelivers, reach
for an adversary.
The most common confusion is to mix up the two "bounds" an algorithm seems to have. When you say your
code "is \Omega(n)", you usually mean that particular code takes at
least linear time — a statement about one algorithm. That is not a problem lower bound.
A problem lower bound quantifies over all algorithms: "no algorithm whatsoever can solve this
in fewer than n-1 comparisons." Showing your favourite method is slow proves
nothing about the problem; someone might have a faster method. The adversary argument is powerful
precisely because it dodges every specific algorithm and argues against the whole class at once.
The second slip: a lower bound is only meaningful relative to a model of operations. "Search
needs n probes" assumes the only move is probing one element. Allow a
different primitive — hashing, or a pre-sorted array — and the floor changes. Always state the model
(comparisons? probes? algebraic operations?) before you state the bound.