Eigenvectors & eigenvalues

A matrix moves every vector — rotating it, stretching it, shearing it. But almost every matrix has a few special directions it cannot rotate: vectors it can only stretch or flip. Those are the eigenvectors, and the stretch factors are the eigenvalues. They are the matrix's skeleton — and the entire reason PCA works.

Av = λvcharacteristic equation power iteration→ PCA

Watch a matrix move a wheel of vectors

Below, a wheel of unit vectors (faint) and where the matrix A sends each one (bold). Drag the sliders and watch: most spokes get rotated — their output points in a different direction than the input. But look for the spokes where input and output line up perfectly. Along those directions the matrix behaves like simple multiplication by a number: Av = λv. No rotation, just scale. That number λ is the eigenvalue; the direction v is the eigenvector.

Find the un-rotatable directions

a (top-left)

b (top-right)

c (bottom-left)

d (bottom-right)

The default matrix is the classic [[1,8],[2,1]]. Its eigen-directions are drawn in green and amber — notice the bold arrows along them sit exactly on top of the direction line, just longer (λ=5: stretched 5×) or flipped and shrunk (λ=−3: reversed and 3×). Now try [[2,0],[0,3]]: a pure axis-scaling — the axes themselves are the eigenvectors, and the eigenvalues are sitting right there on the diagonal. Then try a rotation-like matrix (a=0,b=-1,c=1,d=0): the lines disappear, because a pure rotation rotates everything — its eigenvalues are complex, and there is no real direction it leaves alone.

Where the numbers come from

Asking "which v satisfies Av = λv?" rearranges to (A − λI)v = 0 — a matrix squashing a non-zero vector to zero. From the matrix explainer you know what that means: the matrix A − λI must collapse space, i.e. det(A − λI) = 0. For a 2×2 that determinant is a quadratic in λ — the characteristic equation λ² − tr(A)λ + det(A) = 0. Two roots, two eigenvalues, and each one's eigenvector falls out of the collapsed matrix. For the default matrix:

tr=2, det=1-16=-15 \to λ²-2λ-15=0 \to λ = 5 and -3 where tr(A) = a+d (sum of the diagonal), det(A) = ad-bc, for the default matrix [[1,8],[2,1]] ⓘ characteristic equation on Wikipedia ↗

That's the whole by-hand recipe.

Why repeated multiplication reveals the strongest direction

Feed any vector through A again and again and something remarkable happens: the component along the largest eigenvalue's direction grows fastest (×5 each step here, vs ×3 for the other), so the vector swings toward the dominant eigenvector. This is power iteration — it's how Google's original PageRank found "the important pages" of the web (the dominant eigenvector of the link matrix), and it's the intuition for why long-run behaviour of any repeated linear process is governed by its top eigenvalue.

Power iteration — pull any vector to the dominant direction

0 applications

⚠️ Exam traps: eigenvalues come from det(A−λI)=0, not from the diagonal (unless the matrix is triangular/diagonal) · an eigenvector is a direction — any scalar multiple of it is the same eigenvector · λ can be negative (flip) or zero (that direction is squashed flat — and det(A)=0) · rotations have no real eigenvectors.

Takeaways: eigenvector = direction a matrix only stretches, never rotates · eigenvalue = the stretch factor · found via the characteristic equation λ²−tr·λ+det=0 (2×2) · repeated application converges to the dominant eigenvector (power iteration / PageRank) · the covariance matrix's eigenvectors are exactly PCA's principal components — next.

Curated companion: Setosa — Eigenvectors and Eigenvalues.