ARIMA vs Prophet vs Gradient Boosting
Comparing linear statistical autoregression against additive generalized additive models and tree-based gradient boosting for time series.
Comparison Matrix
ARIMA (p, d, q) PROPHET (Meta) XGBoost / LightGBM
Autoregressive Linear Lags Generalized Additive Model Tabular Supervised Regression
y_t = c + φ_1 y_{t-1} + θ_1 ε_{t-1} y(t) = g(t) + s(t) + h(t) + ε_t y_{t+1} = GBDT(Lags, Rolling, Features)
| Dimension | ARIMA / SARIMAX | Prophet (Meta) | XGBoost / LightGBM |
|---|---|---|---|
| Underlying Model | Linear Autoregressive (p,d,q) | Non-Linear GAM Curve-Fitting | Ensembles of Decision Trees |
| Stationarity Needed? | Yes (Requires differencing d) | No (Handles trend internally) | No (Handles non-linear trends) |
| Missing Data Handling | Fails (Requires continuous time step) | Robust (Curve fitting handles gaps) | Robust (Native missing values) |
| Multiple Seasonality | Complex (SARIMAX gets heavy) | Native (Daily, Weekly, Yearly) | Native (Engineered calendar features) |
| Exogenous Features | Supported (SARIMAX) | Supported (Extra regressors) | Extremely Strong (Hundreds of features) |
| Multi-Item Scaling | 1 model per item (Slow) | 1 model per item (Parallelizable) | 1 global model for 100k SKUs |
Mathematical Formulations
1. ARIMA(p, d, q)
For stationary series $y'_t = \Delta^d y_t$:
$$y't = c + \sum{i=1}^p \phi_i y'{t-i} + \sum{j=1}^q \theta_j \epsilon_{t-j} + \epsilon_t$$
- $p$ (AR): Partial Autocorrelation Function (PACF) determines $p$.
- $d$ (I): Order of differencing needed for stationarity.
- $q$ (MA): Autocorrelation Function (ACF) determines $q$.
2. Prophet Decomposition Model
$$y(t) = g(t) + s(t) + h(t) + \epsilon_t$$
- $g(t)$: Piecewise linear or logistic growth trend.
- $s(t)$: Fourier series modeling seasonality (Weekly $s_W(t)$, Yearly $s_Y(t)$).
- $h(t)$: Holiday effects matrix.
Say this out loud
"ARIMA is a linear statistical model combining autoregressive lags p, differencing d, and moving average error terms q, requiring strict stationarity. Prophet is an additive curve-fitting model robust to missing data and multiple seasonalities. For large-scale enterprise e-commerce with thousands of SKUs and exogenous features (prices, promotions), we use LightGBM/XGBoost to train a single global model across all items."
Follow-ups to expect
- What is DeepAR (Amazon)? An autoregressive recurrent neural network model that learns probabilistic forecasters across thousands of related time series simultaneously, predicting full predictive probability distributions rather than point estimates.
- How do you choose between ARIMA and SARIMAX? SARIMAX adds Seasonal terms $(P, D, Q)_s$ and exogenous regressors $X_t$, essential when modeling data with fixed seasonal periods (e.g. quarterly or monthly cycles).
Check yourself
What do the three parameters (p, d, q) represent in an ARIMA(p, d, q) model?