F-beta & Weighting Errors

What is the F-beta Score?

The standard F1 Score is the harmonic mean of Precision and Recall, giving equal weight to both metrics:

$$F_1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$$

In many real world applications, Precision and Recall are not equally important.

The $F_{\beta}$ Score introduces a weighting parameter $\beta$ to control the relative importance of Recall versus Precision:

$$F_{\beta} = (1 + \beta^2) \times \frac{\text{Precision} \times \text{Recall}}{(\beta^2 \times \text{Precision}) + \text{Recall}}$$

β = 0.5 (F0.5 Score): Weights PRECISION more heavily than Recall.
β = 1.0 (F1 Score):   Weights Precision and Recall EQUALLY.
β = 2.0 (F2 Score):   Weights RECALL more heavily than Precision.

When to Use Which Beta Parameter

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ F0.5 SCORE (β = 0.5)     │ F1 SCORE (β = 1.0)       │ F2 SCORE (β = 2.0)       │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Prioritizes PRECISION.   │ Balanced weight.         │ Prioritizes RECALL.      │
│ False Positives are very │ General baseline for     │ False Negatives are very │
│ costly.                  │ balanced classification. │ dangerous.               │
│ Example: Spam Filtering. │ Example: Document Tagging.│ Example: Medical Diagnosis│
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

1. High Precision Focus ($F_{0.5}$)

False Positives create severe problems. In email spam filtering, sending an important business email to the spam folder (False Positive) is far worse than letting one spam email reach the inbox (False Negative).

2. High Recall Focus ($F_2$)

False Negatives create severe danger. In medical cancer screening or fraud detection, missing an actual cancer patient or fraudulent transaction (False Negative) is far worse than conducting a follow up check on a healthy patient (False Positive).

Why Harmonic Mean Matters

If Precision is $1.0$ and Recall is $0.0$:

• Arithmetic Average: $(1.0 + 0.0) / 2 = 0.50$ (Misleadingly high!).
• Harmonic Mean ($F_1$): $2 \times (1.0 \times 0.0) / (1.0 + 0.0) = 0.0$ (Accurate failure penalty!).

The harmonic mean ensures that if either metric collapses to zero, the combined score drops to zero.

Say this out loud

The F-beta score generalizes the F1 score by adding a weight parameter beta to balance Precision and Recall. Setting beta to 0.5 prioritizes Precision for applications like spam filtering where false positives are costly. Setting beta to 2.0 prioritizes Recall for applications like medical diagnosis where false negatives are dangerous. Harmonic weighting penalizes extreme imbalances.

Followups to expect

1. How does F-beta relate to the PR curve? The F-beta score evaluates a single point on the Precision-Recall curve corresponding to a specific decision threshold.
2. What is the limit of F-beta as beta approaches infinity? As beta approaches infinity, the F-beta score converges entirely to Recall. As beta approaches zero, it converges entirely to Precision.