The Dot Product

Intuition

The dot product measures how much two vectors point in the same direction. Large and positive: they agree; zero: perpendicular; negative: they point away from each other. A linear model's prediction w⋅xw \cdot x is exactly "how well does the input agree with the weights".

Precise explanation

For u,v∈Rnu, v \in \mathbb{R}^n:

u⋅v=∑i=1nuivi.u \cdot v = \sum_{i=1}^{n} u_i v_i.

Geometrically u⋅v=∥u∥ ∥v∥cos⁡θu \cdot v = \lVert u\rVert\,\lVert v\rVert \cos\theta, where θ\theta is the angle between them. It is only defined when both vectors have the same length nn.

Properties: symmetric (u⋅v=v⋅uu\cdot v = v\cdot u), linear in each argument, and u⋅u=∥u∥2≥0u \cdot u = \lVert u\rVert^2 \ge 0.

total = 0
for a, b in zip(u, v):
    total += a * b

Worked example

u=(1,2,3)u = (1, 2, 3), v=(4,−5,6)v = (4, -5, 6): 1⋅4+2⋅(−5)+3⋅6=4−10+18=121\cdot4 + 2\cdot(-5) + 3\cdot6 = 4 - 10 + 18 = 12. (1,0)⋅(0,1)=0(1, 0)\cdot(0, 1) = 0 — perpendicular axes.

Dot product and angle u v θ

Figure (original): u·v = ‖u‖‖v‖cos θ — positive here because θ < 90°.

Common mistakes

  • Using zip without checking lengths: zip silently truncates to the shorter vector.
  • Confusing the dot product (a number) with element-wise multiplication (a vector).
  • Starting the accumulator from u[0]*v[0] and then looping over all elements again.

Why this matters

Dot products are the core operation of linear regression, neural network layers, cosine similarity in NLP retrieval and attention scores in transformers.

References