Dot product & projection

The dot product is the most-used operation in all of ML — it's inside every matrix multiply, every similarity score, every neuron. Underneath the formula u·v = Σ uᵢvᵢ is one geometric idea: how much do two vectors point the same way?

u·v = Σ uᵢvᵢ|u||v|cos θ projectionorthogonality cosine similarity

Two formulas, one number

The dot product can be computed two equivalent ways. Algebraically, multiply matching components and add: u·v = u₁v₁ + u₂v₂. Geometrically, it's |u| |v| cos θ, where θ is the angle between them. Setting them equal is what makes the dot product useful — it converts coordinates into an angle. The sign alone already tells a story:

Positive → the vectors broadly agree (θ < 90°).
Zero → they're exactly perpendicular (θ = 90°). This is the definition of orthogonality.
Negative → they point in opposing directions (θ > 90°).

Move v around and watch the dot product, the angle, and the projection of v onto u (the shadow v casts on u) all update together:

v · x

v · y

u is fixed at (4, 0) — along the x-axis — so the angle is easy to read.

Projection: the dot product's "shadow"

Geometrically, u·v measures how much of v lies along u. If you shine a light straight down onto u, the length of v's shadow is its projection, (u·v)/|u|. That's the precise sense in which the dot product is "how aligned are they": maximal when v points the same way as u, zero when v is perpendicular (no shadow), negative when v leans backwards. Normalising by both lengths gives cosine similarity cos θ = u·v / (|u||v|), the ±1 alignment score used everywhere from word embeddings to recommendation systems (see the embeddings explainer).

Why it's everywhere

A single neuron computes w·x + b — a dot product of weights and inputs — so it literally measures how aligned an input is with the pattern the weights encode. A matrix multiply is just a stack of dot products (each output is one row dotted with the input). Master this one operation and a huge amount of ML stops being mysterious.

Takeaways: u·v = Σ uᵢvᵢ = |u||v|cos θ turns coordinates into an angle. Its sign shows agreement; zero ⟺ perpendicular (orthogonal); it equals v's projection onto u times |u|. Normalised, it's cosine similarity. Neurons and matrix multiplies are built from it.

Curated companion: 3Blue1Brown — "Dot products and duality".