Triplet Loss & Hard Negative Mining

What is Triplet Loss?

Popularized by Google's FaceNet (Schroff et al., 2015), Triplet Loss is a foundational metric learning loss function.

Instead of predicting class probabilities, Triplet Loss trains a neural network $f(x)$ to map images directly into a continuous Euclidean vector space.

It evaluates three images simultaneously:

1. Anchor ($A$): The reference target sample (e.g. Image of Person 1).
2. Positive ($P$): A different image of the SAME class (e.g. Another photo of Person 1).
3. Negative ($N$): An image of a DIFFERENT class (e.g. Photo of Person 2).

  BEFORE TRAINING (Unorganized Space):            AFTER TRIPLET LOSS (Metric Space):
  Anchor A ─── Dist ─── Negative N                Anchor A ──── Pos P  (Pulled Close!)
      │                                                │
    Dist                                               │ Margin alpha
      │                                                ▼
  Positive P                                     Negative N (Pushed Away!)

The Triplet Loss Formula

We want the distance between Anchor and Positive $d(A, P)$ to be smaller than the distance between Anchor and Negative $d(A, N)$ by at least a safety Margin $\alpha$:

$$d(A, P) + \alpha \le d(A, N)$$

$$| f(A) - f(P) |_2^2 + \alpha \le | f(A) - f(N) |_2^2$$

The loss function is:

$$\mathcal{L}(A, P, N) = \max\left( 0, | f(A) - f(P) |_2^2 - | f(A) - f(N) |_2^2 + \alpha \right)$$

• If $d(A, P) + \alpha \le d(A, N) \implies \text{Loss} = 0$ (Target satisfied!).
• If $d(A, P) + \alpha > d(A, N) \implies \text{Loss} > 0$ (Generates non-zero gradients!).

Hard Negative Mining (Why Mining is Critical)

If you randomly pick triplets from a dataset with 10,000 people:

Most random triplets are Easy Triplets: Person 1 vs Person 1 vs Person 999 are already naturally far apart in vector space!

For Easy Triplets:

$$| f(A) - f(P) |^2 - | f(A) - f(N) |^2 + \alpha < 0 \implies \mathbf{\text{Loss} = 0.0}$$

If 99% of your mini-batch generates $0.0$ loss, backpropagation gradients vanish, and training halts!

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. EASY TRIPLETS         │ 2. HARD TRIPLETS         │ 3. SEMI HARD TRIPLETS    │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ d(A, P) + α < d(A, N)    │ d(A, N) < d(A, P)        │ d(A, P) < d(A, N) <      │
│ Loss = 0.0               │ Negative is CLOSER than  │ d(A, P) + α              │
│ Zero gradient signal!    │ Positive! High loss.     │ Best for stable training!│
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

Mining Strategies

• Hard Negative Mining: Select the single farthest Positive $\arg\max_P d(A, P)$ and single closest Negative $\arg\min_N d(A, N)$ within the active mini batch.
• Semi-Hard Negative Mining (FaceNet Standard): Select Negatives $N$ that are farther than Positive $P$, but still lie inside the margin boundary ($d(A, P) < d(A, N) < d(A, P) + \alpha$). This avoids bad local minima caused by noisy outlier labels.

Say this out loud

Triplet Loss trains metric embedding spaces using triplets of Anchor, Positive (same class), and Negative (different class). It pulls Anchor and Positive together while pushing Negative away by margin alpha. Hard Negative Mining selects challenging triplets during mini batch training to prevent zero loss gradient stagnation.

Followups to expect

1. What is Contrastive Loss vs Triplet Loss? Contrastive Loss operates on pairs $(x_1, x_2)$ (same or different). Triplet Loss operates on three samples $(A, P, N)$ simultaneously, providing direct relative distance constraints.
2. Why has Softmax-based Margin Loss (ArcFace) largely replaced Triplet Loss for face recognition? Triplet Loss requires expensive mini batch triplet mining and scales quadratically $O(N^3)$ with dataset size. ArcFace trains using standard classification heads without triplet sampling while achieving superior accuracy.