The Dot Product and Angle

Here's the magic the dot product was hiding. It is also equal to the lengths of the two vectors multiplied together, times the cosine of the angle \theta between them:

\vec{u} \cdot \vec{v} = \lVert \vec{u} \rVert\,\lVert \vec{v} \rVert \cos\theta.

Two formulas, one number. The component version (u_x v_x + u_y v_y) is easy to compute; this version tells you what it means. Rearranged, it hands you the angle between any two vectors:

\cos\theta = \frac{\vec{u} \cdot \vec{v}}{\lVert \vec{u} \rVert\,\lVert \vec{v} \rVert}.

Watch the cosine

Keep both lengths fixed and just swing the angle. When the arrows align (\theta = 0) the dot product is biggest and positive; at a right angle it's 0; pointing opposite ways (\theta = 180^\circ) it's most negative. The dot product is a "same-direction meter."

The sign tells the story

Positive dot product — the angle is sharp (< 90^\circ); the vectors broadly agree.
Zero dot product — a right angle; the vectors are perpendicular.
Negative dot product — the angle is obtuse (> 90^\circ); the vectors broadly oppose.

That one sign — is \vec{u}\cdot\vec{v} positive, zero, or negative? — is how a classifier decides which side of a line a data point falls on. We'll cash that in later.