The Dot Product

Projection and measuring similarity between vectors

Multiplying Vectors?

We know how to add vectors and scale them by numbers. But what does it mean to multiply two vectors together? Unlike regular numbers, there is no single obvious way to do this. The dot product is one answer, and it turns out to be remarkably useful.

The dot product takes two vectors and returns a single number. Not a vector, but a scalar. This number tells us something profound: how much the two vectors point in the same direction.

The Algebraic Definition

Let's start with the computation. Given two vectors a⃗\vec{a} and b⃗\vec{b}, their dot product is simply the sum of the products of their corresponding components:

a⃗⋅b⃗=axbx+ayby\vec{a} \cdot \vec{b} = a_x b_x + a_y b_y

In three dimensions, we just add another term:

a⃗⋅b⃗=axbx+ayby+azbz\vec{a} \cdot \vec{b} = a_x b_x + a_y b_y + a_z b_z

The pattern continues in any number of dimensions. Multiply matching components, then add everything up. Simple enough to compute, but what does it mean?

Interactive: Component-wise Breakdown

3.0
2.0
2.0
3.0
ax⋅bx=3.0×2.0=6.00a_x \cdot b_x = 3.0 \times 2.0 = 6.00
ay⋅by=2.0×3.0=6.00a_y \cdot b_y = 2.0 \times 3.0 = 6.00
a⃗⋅b⃗=6.00+6.00=12.00\vec{a} \cdot \vec{b} = 6.00 + 6.00 = 12.00

Watch how each component contributes to the final dot product. The x-components multiply together, the y-components multiply together, and their sum gives the total. When components have the same sign, they contribute positively. When they have opposite signs, they subtract from the total.

The Geometric Definition

Here is where the dot product becomes beautiful. There is an equivalent geometric formula:

a⃗⋅b⃗=∣a⃗∣∣b⃗∣cos⁡(θ)\vec{a} \cdot \vec{b} = |\vec{a}| |\vec{b}| \cos(\theta)

where θ\theta is the angle between the two vectors, and ∣a⃗∣|\vec{a}| and ∣b⃗∣|\vec{b}| are their lengths (magnitudes).

This formula reveals the geometric soul of the dot product. It measures how aligned two vectors are, scaled by their magnitudes. When vectors point in the same direction (θ=0\theta = 0), the cosine is 1, and the dot product is maximized. When they are perpendicular (θ=90°\theta = 90°), the cosine is 0, so the dot product vanishes. When they point in opposite directions (θ=180°\theta = 180°), the cosine is -1, and the dot product is negative.

Interactive: Angle Between Vectors

45
a⃗⋅b⃗=6.36\vec{a} \cdot \vec{b} = 6.36∣a⃗∣∣b⃗∣=9.00|\vec{a}||\vec{b}| = 9.00
cos⁡(45°)=0.707\cos(45°) = 0.707

Vectors point in similar directions — positive dot product

Notice how the dot product smoothly transitions from positive to zero to negative as the angle increases. The dot product is positive when vectors are less than 90° apart, zero at exactly 90°, and negative when they are more than 90° apart.

Projection: The Heart of the Dot Product

Perhaps the most intuitive way to understand the dot product is through projection. Imagine shining a light directly onto a line containing vector b⃗\vec{b}. The shadow that vector a⃗\vec{a} casts onto this line is called the projection of a⃗\vec{a} onto b⃗\vec{b}.

The length of this shadow, measured as a signed quantity (positive if it points in the same direction as b⃗\vec{b}, negative if opposite), is exactly:

proj length=a⃗⋅b⃗∣b⃗∣=∣a⃗∣cos⁡(θ)\text{proj length} = \frac{\vec{a} \cdot \vec{b}}{|\vec{b}|} = |\vec{a}| \cos(\theta)

If we want the actual projection vector (not just its length), we scale the unit vector in the direction of b⃗\vec{b} by this length:

projb⃗a⃗=a⃗⋅b⃗∣b⃗∣2b⃗\text{proj}_{\vec{b}}\vec{a} = \frac{\vec{a} \cdot \vec{b}}{|\vec{b}|^2} \vec{b}

Interactive: Projection of a onto b

Drag the tip of vector a to see how the projection changes

Projection length: 3.403.40a⃗⋅b^=3.40\vec{a} \cdot \hat{b} = 3.40

Drag vector a⃗\vec{a} around and watch its shadow on b⃗\vec{b}. When a⃗\vec{a} points somewhat in the direction of b⃗\vec{b}, the projection is long and positive. When a⃗\vec{a} is perpendicular to b⃗\vec{b}, the projection vanishes. When a⃗\vec{a} points somewhat opposite to b⃗\vec{b}, the projection becomes negative.

Orthogonality: When the Dot Product Is Zero

Two vectors are orthogonal (perpendicular) if and only if their dot product is zero. This is a direct consequence of the geometric formula: when vectors are at 90° to each other, cos⁡(90°)=0\cos(90°) = 0.

This gives us a quick test for perpendicularity. Instead of measuring angles, just compute the dot product. If it is zero (or very close to zero in practice), the vectors are orthogonal.

Interactive: Orthogonality Detector

Drag either vector tip to explore orthogonality

a⃗⋅b⃗=2.00\vec{a} \cdot \vec{b} = 2.00

Try to make the vectors perpendicular. When you get close, the indicator will turn green and show the right angle symbol. Orthogonality is fundamental in linear algebra, from defining coordinate systems to decomposing signals into independent components.

Why Are Both Definitions Equivalent?

It is not obvious why multiplying and adding components should give the same result as the cosine formula. The connection comes from the law of cosines and some careful algebra.

Consider the vector a⃗−b⃗\vec{a} - \vec{b}. Its length squared, by the law of cosines, is:

∣a⃗−b⃗∣2=∣a⃗∣2+∣b⃗∣2−2∣a⃗∣∣b⃗∣cos⁡(θ)|\vec{a} - \vec{b}|^2 = |\vec{a}|^2 + |\vec{b}|^2 - 2|\vec{a}||\vec{b}|\cos(\theta)

But we can also compute this using components:

∣a⃗−b⃗∣2=(ax−bx)2+(ay−by)2|\vec{a} - \vec{b}|^2 = (a_x - b_x)^2 + (a_y - b_y)^2

Expanding and simplifying both expressions, we find that they are equal if and only if a⃗⋅b⃗=∣a⃗∣∣b⃗∣cos⁡(θ)\vec{a} \cdot \vec{b} = |\vec{a}||\vec{b}|\cos(\theta). The algebraic and geometric definitions are two sides of the same coin.

Applications

The dot product appears everywhere because it captures such a fundamental idea.

Work in physics. When a force F⃗\vec{F} moves an object along a displacement d⃗\vec{d}, the work done is W=F⃗⋅d⃗W = \vec{F} \cdot \vec{d}. Only the component of force in the direction of motion does work. Push perpendicular to motion, and you do no work at all.

Similarity in machine learning. When comparing two vectors of features (say, representing documents or images), the dot product of their normalized versions (the cosine similarity) measures how alike they are. A similarity of 1 means identical directions, 0 means unrelated, and -1 means opposite.

Interactive: Cosine Similarity

30

Similarity=a^⋅b^=0.866\text{Similarity} = \hat{a} \cdot \hat{b} = 0.866

Somewhat similar — vectors point in roughly the same direction

Lighting in graphics. The brightness of a surface depends on the angle between the incoming light and the surface normal. Specifically, the intensity is proportional to the dot product of the light direction and the normal vector. This is why surfaces facing toward light are bright and surfaces facing away are dark.

Detecting direction. In game development, the dot product tells you if an enemy is in front of or behind the player. If the dot product of the forward vector and the direction to the enemy is positive, they are ahead. If negative, they are behind.

Looking Ahead

The dot product becomes essential in later chapters. In orthogonality, we will build entire bases where all pairs of vectors are perpendicular—detected by zero dot products. In eigenvectors, we will find that symmetric matrices have eigenvectors that are mutually orthogonal. The dot product is the tool for measuring and enforcing perpendicularity.

Key Takeaways

  • The dot product has two equivalent definitions: algebraic (sum of component products) and geometric (magnitude times cosine of angle)
  • It measures how much two vectors point in the same direction — positive means similar, zero means perpendicular, negative means opposite
  • Projection is the key geometric interpretation: the dot product with a unit vector gives the shadow length
  • Two vectors are orthogonal (perpendicular) if and only if their dot product is zero
  • Applications span physics (work), machine learning (similarity), graphics (lighting), and game development (direction detection)