Data Augmentation

What is Data Augmentation?

Deep neural networks require massive training datasets to generalize well without overfitting.

When raw data is limited, Data Augmentation creates synthetic new training samples by applying domain-specific transformations to existing data:

  Original Image ("Cat") ──► [ Flip / Crop / Color Jitter ] ──► 5 Augmented Images ("Cat")

Data Augmentation acts as an explicit Regularizer: it teaches the network that class identity should remain invariant to irrelevant variations (like rotation angle, lighting brightness, or synonym word choices).

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. VISION AUGMENTATIONS  │ 2. NLP AUGMENTATIONS     │ 3. AUDIO AUGMENTATIONS   │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Flips, Random Crops,     │ Back-Translation,        │ Time Shifting, Pitch     │
│ Color Jitter, Cutout,    │ EDA (Easy Data           │ Alteration, SpecAugment  │
│ Mixup, AutoAugment.      │ Augmentation), EmbedMix. │ (Masking Spectrograms).  │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

Computer Vision Augmentation Techniques

1. Geometric Spatial Transforms: Random Horizontal Flips, Affine Rotations, Scaling, and Random Cropping.
2. Color Space Transforms: Adjusting Brightness, Contrast, Saturation, and Color Jitter.
3. Erasing & Cutout (DeVries et al., 2017): Randomly masking out rectangular pixel regions, forcing the network to recognize objects from partial clues.
4. AutoAugment / RandAugment (Cubuk et al., 2019): Using automated search or simple random policies to select optimal sequences of augmentation transformations.

NLP Augmentation Techniques

Augmenting discrete text tokens is harder than mutating continuous pixels:

1. Back-Translation: Translate English text to German, then translate German back to English (English -> German -> English). Produces fluent paraphrased sentences.
2. EDA (Easy Data Augmentation - Wei et al., 2019): Synonym Replacement (using WordNet), Random Insertion, Random Swap, and Random Deletion.
3. Contextual Word Substitution: Masking random words in a sentence and using BERT to generate contextually plausible substitute words.

Cardinal Rule of Data Augmentation

  TRAINING SPLIT:     Apply Data Augmentation aggressively to increase diversity!
  VALIDATION / TEST:  NEVER apply augmentation! Keep evaluation data pristine and real-world!

Applying transformations to test sets corrupts ground truth labels and produces invalid evaluation metrics.

(Exception: Test-Time Augmentation (TTA), where predictions across 5 augmented test views are averaged to boost accuracy during inference).

Say this out loud

Data Augmentation artificially expands training data by applying domain specific transformations like crops, flips, back translation, and pitch shifts. It acts as a regularizer, teaching neural networks to stay invariant to irrelevant input changes while preventing overfitting. Augmentation must be applied to training splits only.

Followups to expect

1. What is Test-Time Augmentation (TTA)? Generating predictions for multiple augmented versions of a single test image (e.g. original + horizontal flip + 4 corner crops) and averaging predicted probabilities to improve test accuracy.
2. What is SpecAugment (Park et al., 2019)? A speech recognition augmentation technique that masks out vertical frequency channels and horizontal time blocks directly on log-mel spectrogram images.