Linear Independence

Imagine packing a toolbox with a screwdriver, a second screwdriver identical to the first, and a hammer. That second screwdriver is redundant — take it out and the toolbox can still do everything it could before. Vectors carry exactly the same kind of waste, and spotting it is the whole point of this page.

A vector is redundant in a set if you could have built it out of the others. The vectors \vec{u} and 2\vec{u} carry the same information — the second is no help, because it's already in the span of the first: same direction, just stretched. Two vectors like that are called parallel, and a parallel pair wastes one of its two "ingredient slots" on a direction you already had. Two vectors pointing in genuinely different directions, by contrast, waste nothing — each one reaches somewhere the other alone never could.

A set of vectors is linearly independent when none of them is a linear combination of the others — every vector pulls in a genuinely new direction. If even one is built from the rest, the set is linearly dependent. This single "is anything here wasted?" question turns out to be the backbone of the whole basis-and-dimension story: a basis is exactly a set with zero waste.

The formal test

Eyeballing "does this look redundant?" works fine for two vectors you can draw on paper, but it breaks down fast for three, four, or a hundred vectors in some space you can't picture. So mathematicians use a crisp algebraic test instead. Write a general combination of the vectors and set it equal to the zero vector:

a\,\vec{u} + b\,\vec{v} = \vec{0}

There's always at least one way to hit zero: the boring way, with every weight equal to 0. That combination is always available and never tells you anything. The real question is whether it's the only way.

A set of vectors is linearly independent if the only weights that make the combination equal the zero vector are a = b = \dots = 0.
If some combination with weights that are not all zero also reaches the zero vector, the set is linearly dependent.

Why does a nonzero-weight route to zero always mean redundancy? Because you can rearrange it. If a\,\vec{u} + b\,\vec{v} = \vec{0} with, say, b \neq 0, divide through and solve for \vec{v}:

\vec{v} = -\tfrac{a}{b}\,\vec{u}

Caught red-handed — \vec{v} turns out to be nothing more than a combination of the other vector. Any nonzero solution to the zero-combination equation can always be unpicked this way, which is exactly why it's such a reliable test.

New direction, or not?

Swing \vec{v} around. Whenever it lines up with \vec{u} (same or opposite heading), the two become dependent — \vec{v} is just a scaling of \vec{u}. Every other angle is independent: two directions, no redundancy, enough to span the plane. Watch the caption flip between the two words as you drag.

Three genuinely different directions

Three vectors are linearly independent when none lies in the plane of the other two — then they point in genuinely different directions and together span all of 3-D space. Drag the box below to check that \vec{c} really does lift out of the flat plane containing \vec{a} and \vec{b}: if it instead lay flat inside that plane, the trio would collapse to being dependent.

Worked example: are these two vectors independent?

Test whether \vec{u} = (1, 2) and \vec{v} = (2, 4) are linearly independent. Just glancing at the numbers, notice (2,4) = 2 \cdot (1,2) — the second is exactly twice the first, so they must point along the same line. That's a strong hint they're dependent, but let's confirm it with the formal test rather than trust our eyes alone.

Set up the combination and expand it component by component:

a(1,2) + b(2,4) = (0,0)

This splits into two equations: a + 2b = 0 and 2a + 4b = 0. Look closely and the second equation is simply the first multiplied by 2 — it carries no new information at all. Pick any b, say b = 1, and a = -2 satisfies both equations simultaneously.

We've found a combination with weights that aren't both zero (a=-2,\ b=1) that still lands exactly on \vec{0}. That's the smoking gun: the set is linearly dependent.

Worked example: the standard basis vectors

Now test \vec{u} = (1, 0) and \vec{v} = (0, 1) — pointing straight along the two axes. Set up the very same kind of combination:

a(1,0) + b(0,1) = (0,0)

Component by component this reads a = 0 and b = 0 — directly, with no room to manoeuvre. There is no clever nonzero choice hiding anywhere; the only solution really is the boring one. So this pair is linearly independent. That shouldn't surprise you: they point along perpendicular axes, about as "different a direction" as two vectors in the plane can possibly be.

Worked example: negative scalars still count

A common trap is thinking "scalar multiple" only means a positive stretch. Test \vec{u} = (2, 3) and \vec{v} = (-4, -6). These point in exactly opposite directions, not the same one — surely that makes them independent? Let's check with the formal test rather than guess.

a(2,3) + b(-4,-6) = (0,0)

Expanding gives 2a - 4b = 0 and 3a - 6b = 0 — and once again the second equation is just the first multiplied by 1.5, so it adds nothing new. Choosing b = 1 forces a = 2, and sure enough 2(2,3) + 1(-4,-6) = (0,0) with weights that are not both zero. The set is linearly dependent after all: \vec{v} = -2\vec{u}, a scalar multiple with a negative scalar. "Opposite direction" and "same line" are the same thing in disguise — both point along the one-dimensional span of \vec{u}, just walked in different directions along it.

How many can be independent?

In the plane you can have at most two independent vectors — a third must lie in their span, so it's always redundant. In 3D the limit is three. That ceiling is no accident: it is the dimension of the space, and it is the same number whichever independent vectors you happen to pick. Independence plus spanning is the winning combination we name next.

Here's why a third vector in 2D is always doomed to be redundant. Take any two independent vectors \vec{u}, \vec{v} in the plane — two independent vectors already span the entire plane. So wherever a third vector \vec{w} = (a, b) happens to point, it is already reachable as some combination \vec{w} = c_1\vec{u} + c_2\vec{v}. That's precisely the definition of redundant. Three vectors squeezed into a two-dimensional space simply run out of room to all be "new" — one of them always has to be a combination of the other two.

Worked example: stepping up to three dimensions

If three vectors are automatically dependent in 2D, are they automatically dependent everywhere? No — the ceiling moves with the space. Test \vec{u} = (1,0,0), \vec{v} = (0,1,0), and \vec{w} = (0,0,1) in 3D. Set up the usual combination:

a(1,0,0) + b(0,1,0) + c(0,0,1) = (0,0,0)

Component by component this reads a = 0, b = 0, c = 0 — again no room to manoeuvre, so the trio is linearly independent. Three genuinely different directions fit comfortably in three dimensions, with nothing wasted. Try to add a fourth vector to this set, though, and it collapses right back into the same trap as before: any fourth vector in 3D space is guaranteed to be some combination of these three, for exactly the same reason a third vector was doomed in 2D. The pattern never changes — the number of independent vectors you can ever assemble is capped by the number of dimensions you have to work with, no more and no less.

Recap: the one question to ask

Every example on this page boils down to a single question: does every vector in the set pull its own weight, or could one be quietly assembled from the rest? Parallel pairs fail because one is a stretched (or flipped) copy of the other. A trio in the plane fails because two independent vectors already reach everywhere, leaving no room for a third original direction. The standard basis and the 3D axis vectors pass, because each one points somewhere the others genuinely cannot. Keep asking that one question, and independence stops being a rule to memorise and starts being something you can just see.

Two classic traps catch almost everyone the first time they meet independence:

Independence is a property of the whole set, not of pairs. Check any two vectors out of \vec{u}=(1,0), \vec{v}=(0,1), \vec{w}=(1,1) and no pair is parallel — every two of them point in different directions. Yet the trio of all three together is dependent, because \vec{w} = \vec{u} + \vec{v}. Checking pairs and finding nothing suspicious tells you nothing about the group as a whole; you have to test everyone together.
The zero vector wrecks independence all by itself. If \vec{0} is anywhere in your set, the set is automatically dependent — the combination 1 \cdot \vec{0} + 0 \cdot (\text{everything else}) = \vec{0} uses a nonzero weight (that leading 1) and still hits zero. No other vector needs to look suspicious for this trap to spring; the zero vector alone is enough.

Independence on its own only promises "no waste" — it says nothing about whether your vectors can reach everywhere. A single lonely vector is technically independent (there's nothing else for it to be redundant with!), yet it only reaches a line. The magic happens when you combine independence with spanning: a set that is both independent and spans the whole space is the leanest possible description of that space — every vector earns its place, and nothing is left unreachable. That prized combination gets its own name, a basis, and it's one of the most useful ideas in all of linear algebra: engineers, graphics programmers and physicists are forever hunting for the cleanest possible basis for their problem.

This isn't just an abstract vector-arrow idea. Suppose a scientist records five different measurements from a weather station: temperature, humidity, wind speed, and two separately-wired thermometers that (unsurprisingly) always report almost the same number. Those two thermometer readings behave like near-parallel vectors — one is almost entirely predictable from the other, so together they add barely any new information to the dataset.

Statisticians and engineers have a name for this exact headache: multicollinearity. When the columns of data fed into a model are close to linearly dependent, the model can't tell which of the redundant measurements deserves the credit for an effect, and its predictions become shaky and unstable. The fix is the same idea you've just learned: hunt for a smaller, genuinely independent set of measurements that captures all the same information without the redundancy.

The same principle protects real hardware, not just spreadsheets. A self-driving car might carry a camera, a radar, and a lidar sensor all pointed the same direction — if all three only ever confirmed each other's readings, they'd be nearly as redundant as that pair of thermometers. Engineers deliberately choose sensors that fail in different ways and measure different things (light versus radio waves versus laser pulses) specifically so their combined information stays close to linearly independent — genuinely new evidence from each one, not an echo of what the others already said.