The verb of linear algebra

A matrix, on its own, just sits there — a table of numbers. Feed it a vector, and it comes alive: it eats the vector and hands back a brand new one. That single act — a matrix acting on a vector — is arguably the single most useful operation in all of mathematics. It's how a video game engine spins and moves a character on screen sixty times a second. It's the core step Google's original PageRank algorithm repeats to rank the entire web. It's how a system of several equations in several unknowns gets packaged into one clean question. If matrices are the nouns of linear algebra, matrix-times-vector is the verb — the thing that actually does something.

This page is about learning that one recipe cold: given a matrix A and a vector \vec{x}, how do you compute A\vec{x}, and what does the answer mean?

Here's a taste of just how far the same tiny recipe travels. PageRank's original trick was to represent the entire web as one enormous matrix of links, represent "how much attention each page currently has" as one enormous vector, and repeatedly multiply the matrix by the vector — each multiplication redistributes attention along the links a little more accurately, and after enough rounds the vector settles down into Google's ranking. Billions of web pages, one matrix, one vector, and the exact same "rows dot the vector" arithmetic you're about to do on a 2×2 example by hand.

The recipe: rows dot the vector

Here is the operation everything has been building towards. A matrix multiplies a vector to produce a new vector. The mechanical rule is "rows dot the vector" — each output entry is the dot product of one row of A with \vec{x}:

That works, but it hides the meaning. The real story is this: the output is a linear combination of the matrix's columns, weighted by the entries of the vector:

Both descriptions compute the exact same numbers — "row dot vector" is the fast way to get one entry out by hand, "weighted sum of columns" is the way to actually understand what happened. Keep both in your pocket: the row view for arithmetic, the column view for intuition.

Columns, weighted by the vector

The two faint arrows are the columns of A. The sliders are the entries x and y of the input vector — the weights. The bold arrow is the output A\vec{x}: column one scaled by x, plus column two scaled by y. This single idea — "a matrix mixes its columns" — is the one to carry into everything ahead.

Worked example: a full computation by hand

Let A = \begin{bmatrix} 3 & 1 \\ -2 & 4 \end{bmatrix} and \vec{v} = \begin{bmatrix} 2 \\ 5 \end{bmatrix}. Two rows, two dot products, two output entries. Top entry — row 1 of A dotted with \vec{v}:

Bottom entry — row 2 dotted with \vec{v}:

So A\vec{v} = \begin{bmatrix} 11 \\ 16 \end{bmatrix}. Notice the second row never touched the first calculation, and vice versa — each output entry is its own separate dot product, using its own row and the whole input vector.

Worked example: rotating a vector by 90°

Some matrices have a job title. The matrix R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} is a rotation matrix — multiplying any vector by it spins that vector a quarter turn counter-clockwise, with its length completely unchanged. Watch it work on \vec{v} = \begin{bmatrix} 3 \\ 1 \end{bmatrix}:

Numerically, (3,1) became (-1,3) — swap the coordinates and flip the sign of the new first one. The figure below shows why that's a genuine 90° turn: the original vector and its image are the same length, and a right angle sits exactly between them. Try it on a vector of your own — the pattern (x,y) \mapsto (-y,x) always holds. A full treatment of these "matrices with a job" is coming in rotation matrices.

A quick numerical check confirms the length really is unchanged: the original vector has length \sqrt{3^2+1^2} = \sqrt{10}, and the rotated vector has length \sqrt{(-1)^2+3^2} = \sqrt{1+9} = \sqrt{10} — identical, exactly as a rigid quarter-turn should give. Multiplying by a matrix can stretch, squash, or flip a vector, but this particular matrix only ever turns — never resizes.

Worked example: reading a system of equations as A𝑥 = b

Matrix-vector multiplication also gives you a new way to read a system of equations. Suppose you're told:

Package the left-hand coefficients into a matrix, the unknowns into a vector, and check what multiplying them produces:

That is exactly the left-hand side of the two equations, stacked. So the whole system collapses into the single, compact statement A\vec{x} = \vec{b} where \vec{b} = \begin{bmatrix} 16 \\ 2 \end{bmatrix}. "Solve the system" and "find the vector \vec{x} that this matrix sends to \vec{b}" are the very same question, just phrased two different ways — you can check that x=5,\,y=2 works by plugging back in: 2(5)+3(2)=16 and 5-2=2. Turning that observation into a method for finding \vec{x} from A and \vec{b} is the whole subject of solving linear systems.

Why this is the whole game

Reading A\vec{x} as "weighted columns" explains why matrices are the language of so much. A neuron computes exactly a row dotted with its inputs; a whole layer is one matrix–vector product. And in the next stage we'll see A\vec{x} as a transformation — a matrix is a verb that moves every vector at once.

Notice, too, how much freight one small formula is carrying: the same A\vec{x} = x(\text{col }1) + y(\text{col }2) pattern described spinning a vector 90°, described the left side of two simultaneous equations, and (per row) described what a single neuron computes. Different pictures, different vocabularies, same three keystrokes of arithmetic underneath. That reuse is exactly why this operation earns a page all to itself before anything fancier gets built on top of it.

See it explained