Implement Non-Max Suppression

What is Non-Max Suppression (NMS)?

Computer vision object detectors (YOLO, Faster R-CNN) propose hundreds of candidate bounding boxes around detected objects.

A single car in an image can trigger 30 overlapping bounding boxes across adjacent spatial grid cells.

Non-Max Suppression (NMS) is a greedy post-processing algorithm that filters duplicate detections:

30 Overlapping Candidate Boxes ──► [ NON-MAX SUPPRESSION ] ──► 1 Clean Bounding Box (Highest Confidence!)

Intersection over Union (IoU)

NMS measures box overlap using Intersection over Union (IoU):

$$\text{IoU}(A, B) = \frac{\text{Area}(A \cap B)}{\text{Area}(A \cup B)} = \frac{\text{Area of Overlap}}{\text{Area of Box A} + \text{Area of Box B} - \text{Area of Overlap}}$$

Box A ┌────────┐
      │  ┌─────┼──┐
      └──┼─────┘  │ Box B
         └────────┘
     Overlap Area / Combined Union Area

• $\text{IoU} = 1.0 \implies$ Perfect identical box overlap.
• $\text{IoU} = 0.0 \implies$ No spatial overlap.

The NMS Greedy Algorithm

1. Sort all candidate bounding boxes by confidence score in descending order.
2. Select the box $B^*$ with the highest confidence score and add it to final output list.
3. Compute IoU between $B^*$ and all remaining candidate boxes.
4. Discard any remaining box $B_i$ whose $\text{IoU}(B^*, B_i) > \text{Threshold}$ (for example $0.50$).
5. Repeat until no candidate boxes remain.

NumPy Implementation from Scratch

import numpy as np


class NonMaxSuppressionFromScratch:

  def compute(self, boxes, scores, iou_threshold=0.5):
    # boxes shape: [N, 4] with format (x1, y1, x2, y2)
    # scores shape: [N]
    if len(boxes) == 0:
      return []

    x1 = boxes[:, 0]
    y1 = boxes[:, 1]
    x2 = boxes[:, 2]
    y2 = boxes[:, 3]

    # Calculate areas of all candidate bounding boxes
    areas = (x2 - x1) * (y2 - y1)

    # Sort candidate boxes by confidence score in descending order
    order = np.argsort(-scores)

    keep = []

    while order.size > 0:
      # 1. Pick box with highest confidence score
      i = order[0]
      keep.append(i)

      if order.size == 1:
        break

      # 2. Find intersection coordinates between highest scoring box and rest
      xx1 = np.maximum(x1[i], x1[order[1:]])
      yy1 = np.maximum(y1[i], y1[order[1:]])
      xx2 = np.minimum(x2[i], x2[order[1:]])
      yy2 = np.minimum(y2[i], y2[order[1:]])

      # 3. Calculate width and height of intersection boxes
      w = np.maximum(0.0, xx2 - xx1)
      h = np.maximum(0.0, yy2 - yy1)
      inter = w * h

      # 4. Calculate IoU = Inter / (Area1 + Area2 - Inter)
      iou = inter / (areas[i] + areas[order[1:]] - inter)

      # 5. Keep boxes whose IoU is below threshold
      inds = np.where(iou <= iou_threshold)[0]

      # Shift index by 1 because order[1:] dropped the first element
      order = order[inds + 1]

    return keep

Say this out loud

Non-Max Suppression filters redundant overlapping bounding boxes in object detection. It sorts candidate boxes by confidence score, selects the highest scoring box, and calculates Intersection over Union against remaining candidates. Discarding boxes that exceed IoU thresholds ensures a single clean bounding box is output per object.

Followups to expect

1. What is Soft-NMS? Instead of hard discarding overlapping boxes, Soft-NMS decays their confidence scores proportional to IoU overlap, improving detection when two real objects overlap heavily in crowd scenes.
2. What is Class Aware NMS? Running Non-Max Suppression independently per object class (Car vs Person), preventing a high confidence Car box from accidentally suppressing an overlapping Person box.