Multiplying Matrices
To multiply two matrices, take each row of the left matrix and
dot it with
each column of the right matrix. The entry in row
i, column j of the product is
(AB)_{ij} = (\text{row } i \text{ of } A) \cdot (\text{column } j \text{ of } B).
For this to work the rows of A and columns of
B must be the same length — so the columns of
A must match the rows of B. An
(m\times n) times an (n\times p) gives an
(m\times p). The shared n is consumed.
Row dot column, cell by cell
Step through the four output cells. Each one lights up a row of A and
a column of B, and their dot product fills the answer. Four little dot
products build the whole product.
Why "row dot column"?
It looks arbitrary until you remember what a matrix does. Multiplying by
B then by A means transforming a vector
twice: A(B\vec{x}). The product matrix
AB is the single matrix that does both steps at once — which is
exactly why composing transformations
is matrix multiplication, and why a deep
neural network
is just a chain of matrix products.