MAPE, SMAPE & Forecast Metrics

Why Percentage Errors Matter in Business

Standard metrics like MAE or RMSE measure errors in absolute units (for example off by $$50$).

However, an error of $$50$ means very different things depending on item scale:

• Off by $$50$ on a $$5,000$ Laptop $\implies 1%$ Error (Outstanding Forecast!).
• Off by $$50$ on a $$5$ T-Shirt $\implies 1000%$ Error (Terrible Forecast!).

Percentage Forecast Metrics evaluate errors relative to target magnitude, allowing business leaders to compare forecasting performance across distinct product lines.

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. MAPE (Standard)       │ 2. SMAPE (Symmetric)     │ 3. WMAPE (Weighted Volume│
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Mean absolute percentage │ Symmetric bound scaling  │ Sum of errors divided by │
│ error relative to target.│ using combined target and│ sum of total actual sales│
│ Easy to explain!         │ prediction average denominator| volume. Retail standard! │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

1. Mean Absolute Percentage Error (MAPE)

$$\text{MAPE} = \frac{100%}{N} \sum_{i=1}^N \left| \frac{y_i - \hat{y}_i}{y_i} \right|$$

• Pros: Scale independent. Expresses error as an intuitive percentage (for example "Our demand forecasts are off by $8.2%$ on average").
• Cons (Division by Zero): If actual value $y_i = 0$, MAPE divides by zero!
• Cons (Asymmetry Penalty): MAPE penalizes over-predictions far more heavily than under-predictions.

2. Symmetric MAPE (SMAPE)

Fixes MAPE asymmetry by placing the average of actual $y_i$ and predicted $\hat{y}_i$ in the denominator:

$$\text{SMAPE} = \frac{100%}{N} \sum_{i=1}^N \frac{|y_i - \hat{y}_i|}{(|y_i| + |\hat{y}_i|) / 2}$$

SMAPE bounds individual percentage errors between $0%$ and $200%$, preventing extreme division spikes.

3. Weighted MAPE (WMAPE / MAD over Mean)

Used heavily in retail demand forecasting (Amazon, Walmart) to handle items with zero sales days:

$$\text{WMAPE} = \frac{\sum_{i=1}^N |y_i - \hat{y}i|}{\sum{i=1}^N y_i}$$

Instead of dividing per item, WMAPE sums absolute errors across all products and divides by total actual sales volume.

High volume items naturally contribute more weight to overall accuracy, and zero sales items cause zero division failures!

Say this out loud

MAPE, SMAPE, and WMAPE evaluate forecasting accuracy in relative percentage terms. MAPE measures relative percentage error, providing an intuitive business metric across different sales scales. SMAPE bounds percentage errors symmetrically. WMAPE solves division by zero issues on sparse zero sales items by dividing total absolute error by total actual sales volume.

Followups to expect

1. What is Mean Absolute Scaled Error (MASE)? A time series metric comparing forecast errors against a simple naive baseline forecast (predicting yesterday's sales), suitable for seasonal and zero sales demand series.
2. Why does optimizing MAE produce median forecasts while MAPE produces mode forecasts? Minimizing MAE loss predicts the median of target distribution, whereas minimizing MAPE loss biases predictions toward lower values to reduce percentage penalties.