Reading a Confusion Matrix

What is a Confusion Matrix?

Accuracy only tells you how often a classifier is correct. It does not tell you how the model is failing.

A Confusion Matrix is a 2x2 grid (for binary classification) that breaks down model predictions against true ground truth labels:

                      ACTUAL GROUND TRUTH
                     Positive          Negative
PREDICTED   Positive │ True Positive  │ False Positive │
          ───────────┼────────────────┼────────────────┤
            Negative │ False Negative │ True Negative  │

Understanding the 4 Quadrants

1. True Positive (TP): Model predicted Positive, and actual label is Positive (Correct!). Example: Sick patient correctly diagnosed as sick.
2. False Positive (FP - Type I Error): Model predicted Positive, but actual label is Negative (False Alarm!). Example: Healthy patient incorrectly diagnosed as sick.
3. False Negative (FN - Type II Error): Model predicted Negative, but actual label is Positive (Missed Case!). Example: Sick patient incorrectly sent home as healthy.
4. True Negative (TN): Model predicted Negative, and actual label is Negative (Correct!). Example: Healthy patient correctly diagnosed as healthy.

Deriving Key Metrics from the Matrix

┌─────────────────────────────────────────────────────────────┐
│ Accuracy    = (TP + TN) / (TP + FP + FN + TN)               │
│ Precision   = TP / (TP + FP)   [Quality of Positive Claims] │
│ Recall      = TP / (TP + FN)   [Coverage of Actual Positives]│
│ Specificity = TN / (TN + FP)   [Coverage of Actual Negatives]│
└─────────────────────────────────────────────────────────────┘

Extending to Multi Class Problems

For an $N$ class problem (for example classifying dogs, cats, and birds), the confusion matrix expands to an $N \times N$ grid:

             Actual Dog   Actual Cat   Actual Bird
Predicted Dog    45           2            1
Predicted Cat     3          40            5
Predicted Bird    0           1           48

• Diagonal Cells (Top-Left to Bottom-Right): Correct predictions.
• Off-Diagonal Cells: Specific classification confusions (for example 3 Actual Dogs incorrectly classified as Cats).

Say this out loud

A Confusion Matrix breaks down classification predictions into True Positives, False Positives, False Negatives, and True Negatives. It reveals specific error types that raw accuracy masks, forming the mathematical foundation for calculating Precision, Recall, Specificity, and F1 score across binary and multi class classification tasks.

Followups to expect

1. Which error type is worse in fraud detection vs spam filtering? In fraud detection, False Negatives (missing real fraud) are worse. In spam filtering, False Positives (sending a legitimate email to spam) are worse.
2. What is a Normalized Confusion Matrix? A confusion matrix where values in each row are divided by the total number of actual ground truth samples in that class, converting raw counts into percentages.