Face Recognition & Metric Learning

Closed-Set Classification vs Open-Set Verification

Standard image classifiers map inputs to fixed output classes:

Predict class $y \in {\text{Cat}, \text{Dog}, \text{Car}}$

This is Closed-Set Classification.

Face Recognition is an Open-Set Verification problem:

1. Systems must verify new employees or users whose faces were never seen during training.
2. Retraining a 100-layer neural network every time a new employee joins a company is impossible.

Instead of outputting class labels, Face Recognition models output a 128-d or 512-d Face Embedding Vector ($v$):

  Input Face Image A ──► [ DEEP METRIC BACKBONE ] ──► Embedding Vector u [1 x 512]
  Input Face Image B ──► [ DEEP METRIC BACKBONE ] ──► Embedding Vector v [1 x 512]

  Decision Rule:
  - If Cosine Similarity(u, v) > 0.75 ──► SAME PERSON!
  - If Cosine Similarity(u, v) < 0.75 ──► DIFFERENT PEOPLE!

Deep Metric Learning & Margin Losses

To make vector distance thresholding work, training must enforce two properties in embedding space:

1. Intra-Class Compactness: Embeddings of the SAME person must cluster tightly together.
2. Inter-Class Discrepancy: Embeddings of DIFFERENT people must be pushed far apart.

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. TRIPLET LOSS (2015)   │ 2. COSFACE (2018)        │ 3. ARCFACE (2019)        │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Compares (Anchor,        │ Adds cosine margin       │ Adds angular margin m    │
│ Positive, Negative)      │ penalty m to target      │ inside cosine function:  │
│ triplets. Hard sampling! │ logits: cos(θ) - m.      │ cos(θ + m). SOTA!        │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

ArcFace (Additive Angular Margin Loss - Deng et al., 2019)

ArcFace is the state-of-the-art loss function for face recognition.

It normalizes both weight vectors $W_j$ and feature vectors $x_i$ so their dot product depends purely on the angle $\theta_j$ between them:

$$W_j^T x_i = |W_j| |x_i| \cos \theta_j = \cos \theta_j$$

ArcFace adds an Additive Angular Margin $m$ (e.g. $m = 0.5$ radians) to the target angle $\theta_{y_i}$:

$$\mathcal{L}{\text{ArcFace}} = -\log \frac{e^{s \cdot \cos(\theta{y_i} + m)}}{e^{s \cdot \cos(\theta_{y_i} + m)} + \sum_{j \neq y_i} e^{s \cdot \cos \theta_j}}$$

• $s$: Feature scale factor (e.g. $s = 64$).
• $m$: Angular margin penalty.

By penalizing the target angle with $+m$, ArcFace forces the network to pull feature vectors of the same identity into an extremely tight angular cone on a hyper-sphere!

Say this out loud

Face Recognition uses Metric Learning to map facial images into a dense vector embedding space for open set verification. Instead of closed set classification, face models evaluate identity by computing cosine similarity between embedding vectors against a threshold. Modern ArcFace loss adds an angular margin m to target feature angles, enforcing tight hyper spherical intra class clustering.

Followups to expect

1. What is Face Alignment / Detection preprocessing? Using a detector (like RetinaFace or MTCNN) to locate facial landmarks (eyes, nose, mouth corners) and warp/crop the face into a normalized canonical orientation before embedding extraction.
2. What is LFW (Labeled Faces in the Wild)? A classic benchmark dataset containing 13,000 face images used to evaluate open set face verification accuracy.