Dot Product & Cosine Similarity

Dot Product vs Cosine Similarity

The Dot Product multiplies corresponding elements and sums the results:

u . v = sum( u_i * v_i ) = ||u|| * ||v|| * cos(theta)

The Cosine Similarity isolates the angle theta by dividing out vector lengths:

cos(theta) = ( u . v ) / ( ||u|| * ||v|| )

┌──────────────────────────┬──────────────────────────┐
│  1. DOT PRODUCT          │  2. COSINE SIMILARITY    │
├──────────────────────────┼──────────────────────────┤
│ Combines angle AND length│ Measures PURE angle      │
│ Range: (-inf, +inf)      │ Range: [-1, 1]           │
│ Favors long vectors      │ Scale-invariant          │
└──────────────────────────┴──────────────────────────┘

Why Length Normalization Saves GPU Time

In vector databases and recommendation systems, calculating Cosine Similarity over millions of vectors requires dividing by vector norms millions of times.

Smart engineering fix: Pre-normalize all vectors to unit length (||u|| = 1) during data ingestion.

When vectors have unit length:

Cosine Similarity = u . v / (1 * 1) = u . v

Normalizing vectors upfront converts slow similarity math into lightning-fast GPU matrix multiplications.

When to Use Which Metric

1. Document Classification (TF-IDF): Use Cosine Similarity. Longer documents naturally contain more words (larger vector magnitude), but should still match short documents with identical word proportions.
2. Matrix Factorization (RecSys): Use Dot Product. Vector magnitude carries meaningful user interaction intensity or item popularity signal.
3. Deep Embeddings (Sentence Transformers): Pre-normalize vectors and use Dot Product for sub-10ms vector database retrieval.

Say this out loud

Dot product measures vector alignment weighted by length. Cosine similarity isolates directional angle by dividing by vector magnitudes, scaling results from -1 to +1. When vectors are pre-normalized to unit length, Dot Product becomes identical to Cosine Similarity, enabling fast hardware dot products in vector databases.

Follow-ups to expect

• What is the geometric relationship between Euclidean distance and Cosine similarity for unit vectors? For unit vectors, squared Euclidean distance is directly related to Cosine similarity: ||u - v||^2 = 2 - 2*(u . v).
• What does a Cosine similarity of -1 mean? It means the vectors point in exact opposite directions (180 degrees apart in vector space).