Mixup & CutMix

Advanced Image Regularization

Standard data augmentation techniques (flips, crops, rotations) alter a single image while keeping its class label fixed.

However, deep neural networks tend to memorize exact training boundaries, becoming overconfident on out-of-distribution inputs.

Mixup and CutMix regularize models by blending two different training images and their target labels together:

┌──────────────────────────┬──────────────────────────┐
│ 1. MIXUP (Zhang 2017)    │ 2. CUTMIX (Yun 2019)     │
├──────────────────────────┼──────────────────────────┤
│ Translucent pixel blend. │ Pastes rectangular patch │
│ x = λ*x1 + (1-λ)*x2      │ from Image B onto Image A.│
│ Linear label interpolation│ Soft labels proportional │
│ y = λ*y1 + (1-λ)*y2      │ to patch area ratio.     │
└──────────────────────────┴──────────────────────────┘

1. Mixup (Zhang et al., 2018)

Mixup blends two random image samples $(x_1, y_1)$ and $(x_2, y_2)$ using mixing ratio $\lambda \sim \text{Beta}(\alpha, \alpha)$:

$$\tilde{x} = \lambda x_1 + (1 - \lambda) x_2$$

$$\tilde{y} = \lambda y_1 + (1 - \lambda) y_2$$

  Image 1 ("Dog", Label = [1, 0]) ──┐
                                     ├──► λ = 0.7 ──► Translucent Ghost Overlay Image!
  Image 2 ("Cat", Label = [0, 1]) ──┘                Target Label: [0.70 Dog, 0.30 Cat]

Why Mixup Works

Forces the neural network to favor linear behavior between training samples.

Instead of sharp step decision boundaries, Mixup smooths out network transition spaces, making predictions robust against adversarial noise.

2. CutMix (Yun et al., 2019)

Mixup creates ghost-like translucent images that look unnatural to human eyes.

CutMix cuts a rectangular patch from Image 2 and pastes it onto Image 1:

  ┌──────────────────┐        ┌──────────────────┐        ┌──────────────────┐
  │                  │        │                  │        │        ┌───┐     │  Target Label:
  │    Image 1       │   +    │    Image 2       │   ──►  │ Image 1│Img│     │  0.75 * Dog
  │    ("Dog")       │        │    ("Cat")       │        │        └───┘     │  + 0.25 * Cat
  └──────────────────┘        └──────────────────┘        └──────────────────┘

The target label is weighted by the exact pixel area fraction $M$ of the pasted patch:

$$\tilde{x} = M \odot x_1 + (\mathbf{1} - M) \odot x_2$$

$$\tilde{y} = \lambda y_1 + (1 - \lambda) y_2, \quad \text{where } \lambda = 1 - \frac{\text{Area(Patch)}}{\text{Area(Image)}}$$

Why CutMix Outperforms Mixup

1. Preserves Spatial Locality: Pasted patches contain natural, un-blurred pixel textures.
2. Eliminates Overconfidence: Forces the network to identify objects from partial regional views rather than relying on a single central feature.

Say this out loud

Mixup and CutMix blend pairs of training images and target labels to regularize deep vision models. Mixup creates convex linear pixel overlays, smoothing decision boundaries. CutMix replaces a rectangular patch of image A with a patch from image B, setting target labels equal to patch area proportions to force models to learn from partial object views.

Followups to expect

1. What value of alpha is typically chosen for the Beta distribution? $\alpha = 0.2$ to $1.0$. Setting $\alpha = 1.0$ generates a uniform distribution $U(0, 1)$ for mixing ratio $\lambda$.
2. Can Mixup be applied to text embeddings? Yes, EmbedMix applies Mixup interpolation to word or sentence embedding vectors in hidden layers rather than raw text tokens.