Resizing, Normalization & Colour Spaces

The Computer Vision Data Pipeline

Deep neural networks do not accept raw, arbitrary camera files.

Before feeding images into a vision model, inputs must undergo Image Preprocessing to standardize spatial dimensions, channel orders, and pixel distributions:

  Raw Camera Image (1920x1080, BGR, [0..255])
                    │
                    ▼
  [ 1. Color Conversion (BGR -> RGB) ]
                    │
                    ▼
  [ 2. Aspect-Preserving Resize & Letterbox (224x224) ]
                    │
                    ▼
  [ 3. Scale Pixels to Float Range [0.0, 1.0] ]
                    │
                    ▼
  [ 4. Channel Normalization (ImageNet μ, σ) ] ──► Standardized Input Tensor [1 x 3 x 224 x 224]

1. Resizing & Aspect Ratio Preservation

Computer vision models require fixed input tensor dimensions (e.g. $224 \times 224$ or $640 \times 640$).

The Naive Stretch Trap

Resizing a $1920 \times 1080$ rectangular image directly to $224 \times 224$ stretches objects vertically, distorting geometric shapes.

┌──────────────────────────┬──────────────────────────┐
│ A. CENTER CROPPING       │ B. LETTERBOXING (PADDING)│
├──────────────────────────┼──────────────────────────┤
│ Crops central 224x224    │ Resizes image keeping    │
│ region. Preserves aspect │ aspect ratio, filling    │
│ ratio, but discards outer│ remaining borders with   │
│ background pixels!       │ gray/black padding pixels│
└──────────────────────────┴──────────────────────────┘

Used in YOLO object detection, Letterboxing scales the long edge to fit target dimensions and pads remaining borders.

2. Color Spaces & Library Traps

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ COLOR SPACE              │ CHANNELS                 │ PRIMARY USE CASE         │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ RGB                      │ Red, Green, Blue         │ PyTorch, PIL, Torchvision│
│ BGR                      │ Blue, Green, Red         │ OpenCV (cv2.imread!)     │
│ HSV                      │ Hue, Saturation, Value   │ Color-based segmentation │
│ Lab                      │ Lightness, a, b          │ Perceptual distance      │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

The OpenCV BGR Trap

cv2.imread() loads images in BGR order, while PyTorch models expect RGB order!

Passing BGR images directly into a pretrained PyTorch model swaps Red and Blue channels, causing severe accuracy drops.

Fix: cv2.cvtColor(image, cv2.COLOR_BGR2RGB).

3. Pixel Scaling & Channel Normalization

Standard raw images store uint8 pixel values in range $[0, 255]$.

1. Scale to $[0.0, 1.0]$: Divide raw pixels by $255.0$.
2. Channel Normalization: Standardize channels using pretraining statistics:

$$x_{\text{normalized}} = \frac{x - \mu}{\sigma}$$

For ImageNet pretrained backbones (ResNet, ViT):

• Mean $\mu$: [0.485, 0.456, 0.406]
• Std $\sigma$: [0.229, 0.224, 0.225]

Normalizing ensures input feature distributions match the exact statistical distributions expected by pretrained layer weights.

Say this out loud

Image Preprocessing converts raw pixels into standardized tensors. Letterboxing preserves geometric aspect ratios by padding borders. OpenCV loads images in BGR format, requiring conversion to RGB for PyTorch. Inputs are scaled to 0 to 1 and normalized using ImageNet mean and std vectors to match pretraining weight statistics.

Followups to expect

1. What interpolation method is best for image downsampling vs upsampling? Use Area Interpolation (INTER_AREA) for downsampling to prevent aliasing artifacts. Use Bilinear or Bicubic Interpolation (INTER_CUBIC) for upsampling.
2. Why do vision transformers (ViT) require strict fixed spatial input sizes? ViT adds learnable 1D positional embeddings corresponding to fixed patch grid positions ($14 \times 14 = 196$ patches for $224 \times 224$ images). Changing resolution requires 2D bicubic interpolation of positional embeddings.