Image Augmentation Strategies

Advanced Vision Augmentation Techniques

  TRADITIONAL AUGMENTATION         MIXUP (Convex Blend)             CUTMIX (Patch Paste)
  (Crop, Flip, Rotate)             λ = 0.6 Cat, 0.4 Dog             60% Cat Area, 40% Dog Area
  ┌──────────────────────┐         ┌──────────────────────┐         ┌──────────────────────┐
  │ [ Rotated Dog ]      │         │ [ Semi-transparent ] │         │ [ Cat Image ]        │
  │                      │         │ [ Blend Cat + Dog  ] │         │  ┌──────────┐        │
  │                      │         │                      │         │  │ Dog Patch│        │
  └──────────────────────┘         └──────────────────────┘         └──┴──────────┴────────┘
  Label: 100% Dog                  Label: 0.6 Cat + 0.4 Dog         Label: 0.6 Cat + 0.4 Dog

1. Mixup (Zhang et al., 2017)

Mixup trains neural networks on linear convex combinations of pairs of examples:

$$\tilde{x} = \lambda x_A + (1 - \lambda) x_B$$

$$\tilde{y} = \lambda y_A + (1 - \lambda) y_B$$

Where $\lambda \sim \text{Beta}(\alpha, \alpha)$ for hyperparameter $\alpha \in [0.2, 1.0]$.

Why Mixup Works

Traditional cross-entropy forces models to output extreme 100% confidence predictions right up to the decision boundary.

Mixup regularizes the network to behave linearly in-between training samples, suppressing overconfident predictions on out-of-distribution inputs.

2. CutMix (Yun et al., 2019)

Mixup produces unnatural semi-transparent "ghost" images.

CutMix cuts a rectangular patch from image $B$ and pastes it onto image $A$:

$$\tilde{x} = \mathbf{M} \odot x_A + (\mathbf{1} - \mathbf{M}) \odot x_B$$

$$\tilde{y} = \lambda y_A + (1 - \lambda) y_B$$

Where binary mask $\mathbf{M} \in {0, 1}^{H \times W}$ sets patch area ratio equal to $\lambda = 1 - \frac{\text{BoxArea}}{H \cdot W}$.

Why CutMix Outperforms Mixup

• Preserves natural pixel statistics (no ghosting artifacts).
• Forces the model to identify objects from partial regional cues, improving localization and robustness against occlusion.

3. RandAugment (Cubuk et al., 2020)

Prior methods (AutoAugment) used expensive Reinforcement Learning to search over thousands of augmentation combinations.

RandAugment reduces the search space to just two global hyperparameters:

1. $N$: Number of augmentation transformations to apply sequentially per image (e.g. $N = 2$).
2. $M$: Global magnitude parameter controlling transformation intensity ($M \in [1, 30]$).

# RandAugment PyTorch Implementation
from torchvision.transforms import v2

transforms = v2.Compose([
    v2.RandAugment(num_ops=2, magnitude=9), # N=2, M=9
    v2.ToTensor()
])

Say this out loud

Advanced data augmentation regularizes vision models and prevents overfitting. Mixup linearly blends pairs of images and labels. CutMix pastes realistic image patches from one image into another, setting target label proportions to patch area and forcing models to learn local regional features. RandAugment simplifies automated augmentation by controlling operations with just two hyperparameters: number of ops N and magnitude M.

Follow-ups to expect

• What is Label Smoothing? Replacing hard one-hot target vectors $[0, 1, 0]$ with softened target vectors $[0.05, 0.90, 0.05]$, preventing the model from becoming overconfident and improving calibration.
• How does CutMix affect Object Detection? CutMix works exceptionally well for object detection: pasting object patches into new background scenes increases training sample density for rare object classes.