Time Series Forecasting Basics

Predicting future values of time-indexed data by modeling trend, seasonality, autocorrelation, and exogenous features.

🟡 intermediate5 min readtime-series

Time Series Forecasting predicts future observations y_{t+h} given historical observations y_1, ..., y_t. Time series data decomposes into Trend T_t, Seasonality S_t, Cyclical patterns C_t, and Residual Noise I_t. Classical statistical approaches (ARIMA, Exponential Smoothing) rely on stationarity and autocorrelation. Modern Machine Learning approaches (XGBoost, Prophet, DeepAR, Temporal Fusion Transformers) frame forecasting as supervised regression using lagged features and rolling window statistics.

Time Series Components & Decomposition

Every time series $Y_t$ can be decomposed into 4 components:

Additive Model:        Y_t = Trend (T_t) + Seasonality (S_t) + Cyclical (C_t) + Irregular Noise (I_t)

Multiplicative Model:  Y_t = Trend (T_t) × Seasonality (S_t) × Cyclical (C_t) × Irregular Noise (I_t)

     Additive (Constant Amplitude)             Multiplicative (Growing Amplitude)
       /\    /\    /\    /\                       /\      /\       /\        /\
      /  \  /  \  /  \  /  \                     /  \    /  \     /  \      /  \
     /    \/    \/    \/    \                   /    \  /    \   /    \    /    \
                                               /      \/      \ /      \  /      \

Additive: Use when seasonal variation amplitude remains constant as trend increases.
Multiplicative: Use when seasonal fluctuations expand or shrink proportionally with rising trend (convert to additive via $\ln(Y_t)$).

Supervised ML Feature Engineering Pipeline

To forecast $y_{t+1}$ using GBDT models (XGBoost, LightGBM):

Target: y_{t+1}
Features:
  - Lag Features:       y_t, y_{t-1}, y_{t-7}, y_{t-365}
  - Rolling Windows:    Mean(y_{t-7..t}), Std(y_{t-7..t}), Max(y_{t-30..t})
  - Calendar Features:   DayOfWeek, IsWeekend, Month, Quarter, IsHoliday
  - Exogenous Features: Price, Promotion, Temperature, Marketing Spend

# Create Lag Features in Pandas
df['lag_1'] = df['y'].shift(1)
df['lag_7'] = df['y'].shift(7)
df['rolling_mean_7'] = df['y'].shift(1).rolling(7).mean()

Cross-Validation Protocol: TimeSeriesSplit

NEVER use standard random K-Fold!

Use Expanding Window or Sliding Window temporal splits to prevent Look-Ahead Leakage:

Fold 1:  [ Train: Train Data (Jan-Jun) ] ──► [ Val: July ]
Fold 2:  [ Train: Train Data (Jan-Jul) ] ──► [ Val: August ]
Fold 3:  [ Train: Train Data (Jan-Aug) ] ──► [ Val: September ]

Say this out loud

"Time series forecasting models trend, seasonality, and residual noise. For GBDT models like XGBoost, we frame forecasting as supervised regression by engineering lag features, rolling window statistics, and calendar variables. We must strictly use Expanding Window temporal cross-validation (TimeSeriesSplit) to prevent look-ahead data leakage from shuffling future observations into training sets."

Follow-ups to expect

What is Direct vs Recursive Multi-Step Forecasting? Direct forecasting trains separate independent models for each future step $h$ ($\hat{y}{t+1}, \hat{y}{t+2}$). Recursive forecasting uses a single model to predict $t+1$, plugging predicted $\hat{y}_{t+1}$ back into the model as a pseudo-lag feature to predict $t+2$.
What is Autoregressive Conditional Heteroskedasticity (ARCH)? A statistical model used in financial forecasting to model time-varying volatility and variance clusters in stock returns.

Check yourself

Question 1 of 3

Why must cross-validation for time series forecasting use Temporal Splitting (e.g. TimeSeriesSplit / Expansion Window) rather than standard K-Fold random sampling?

More in Classical ML

See all →

Bias–Variance Tradeoff4 min Overfitting vs Underfitting3 min Linear Regression4 min