MSE, MAE, RMSE, R²

Evaluating Continuous Numeric Predictions

Unlike classification (where predictions are discrete classes), Regression Models predict continuous numbers (for example house prices, delivery times, stock values).

Evaluating continuous predictions requires measuring the magnitude of error residuals:

$$\text{Residual } e_i = y_i - \hat{y}_i$$

┌──────────────────────────┬──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. MAE (Abs Error)       │ 2. MSE (Squared Error)   │ 3. RMSE (Root Squared)   │ 4. R-SQUARED (Variance)  │
├──────────────────────────┼──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Linear mean error.       │ Quadratic mean error.    │ Square root of MSE.      │ Fraction of target       │
│ Robust to outliers!      │ Heavily penalizes large  │ Same units as target,    │ variance explained by    │
│                          │ outlier errors!          │ penalizes outliers!      │ the model (0 to 1).      │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

1. Mean Absolute Error (MAE)

$$\text{MAE} = \frac{1}{N} \sum_{i=1}^N |y_i - \hat{y}_i|$$

• Pros: Easy to interpret. An MAE of $5.0$ in delivery prediction means predictions are off by 5 minutes on average.
• Robustness: Treats all errors linearly. Outliers do not distort the overall metric.

2. Mean Squared Error (MSE) and Root Mean Squared Error (RMSE)

$$\text{MSE} = \frac{1}{N} \sum_{i=1}^N (y_i - \hat{y}_i)^2, \quad \text{RMSE} = \sqrt{\text{MSE}}$$

• Outlier Penalty: Squaring errors means an error of $10$ adds $100$ to the loss, while an error of $2$ adds only $4$.
• RMSE Advantage: Taking the square root converts squared units back into original target units (for example dollars instead of squared dollars).

3. R-squared ($R^2$ Coefficient of Determination)

$$R^2 = 1 - \frac{\sum (y_i - \hat{y}i)^2}{\sum (y_i - \bar{y})^2} = 1 - \frac{\text{SS}{\text{res}}}{\text{SS}_{\text{tot}}}$$

• Baseline Comparison: Compares model errors against a naive baseline model that simply predicts the mean target value $\bar{y}$ everywhere.
• Interpretation: $R^2 = 0.85$ means the model explains $85%$ of the total target variance. $R^2 = 0.0$ means the model is no better than predicting the average value everywhere.

Summary Metric Selection Guide

Say this out loud

Regression metrics evaluate continuous numeric predictions. MAE measures average absolute error linearly and is robust against outliers. MSE and RMSE square errors, penalizing large prediction deviations heavily. RMSE returns error values to the original target units. R-squared measures the proportion of target variance explained by model features.

Followups to expect

1. Can R-squared be negative? Yes, R-squared becomes negative if the trained model performs worse than a simple horizontal line predicting the mean target value everywhere on held out test data.
2. What is Huber Loss? A loss function combining MAE and MSE: behaving like MSE for small errors and switching to MAE for large errors, balancing smoothness with outlier robustness.