Missing Data & Imputation

Understanding missing data mechanisms (MCAR, MAR, MNAR) and selecting robust imputation strategies.

🟢 beginner4 min readdata

Missing data degrades model quality and causes pipeline failures if unhandled. Missingness falls into three mechanisms: Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR). Imputation techniques range from simple Mean/Median/Mode insertion and indicator flags to Iterative MICE (Multivariate Imputation by Chained Equations) and KNN imputation. Modern GBDTs (XGBoost, LightGBM) handle missing values natively during split finding.

The 3 Missing Data Mechanisms

                         Missing Data Mechanisms
    ┌───────────────────────────┬───────────────────────────┬───────────────────────────┐
    ▼                           ▼                           ▼                           ▼
  MCAR (Completely at Random)      MAR (At Random)             MNAR (Not at Random)
- Missingness is 100% random.   - Missingness depends on    - Missingness depends on the
- Unrelated to observed or        OBSERVED features (e.g.     UNOBSERVED value itself
  unobserved values.              Younger users skip income).  (High income skips income).
- Safe to drop / impute.        - Conditional imputation ok.- Informative missingness!

Imputation Matrix

Strategy	Mechanics	Model Suitability	Pros & Cons
Drop Rows (Complete Case)	Delete any row with $\ge 1$ NaN	Statistical tests	Wastes data; biases sample if not MCAR
Mean / Median / Mode	Replace NaN with constant	Linear models, Neural Nets	Fast, simple. Con: Distorts variance & covariance
Missing Indicator Flag	Add binary column `x_is_missing = 1`	All models	Preserves informative MNAR missingness signal
KNN Imputation	Replace NaN with average of $k$ nearest rows	Small/Medium datasets	Captures feature interactions. Con: $O(N \cdot d)$ slow
MICE (Iterative SVD)	Regress feature $j$ on all other features iteratively	Linear models, Healthcare	Gold-standard accuracy. Con: Computationally heavy
Native GBDT Branching	Send NaNs to branch yielding max split gain	XGBoost, LightGBM	Zero pre-processing overhead; learns optimal split

Best Practice Pipeline for Linear / Neural Models

Create binary indicator column: x_missing_flag = IS_NULL(x).
Impute x using training fold Median (continuous) or Mode (categorical).
Scale features using StandardScaler.

Never impute missing values using global dataset statistics! Compute medians strictly within cross-validation training folds.

Say this out loud

"Missing data mechanisms dictate imputation: MCAR is completely random, MAR depends on observed features, and MNAR is informative missingness depending on unobserved values. For linear models and neural nets, we create explicit missing indicator flags and impute with median. For GBDTs like XGBoost, we leverage native missing value branching, which automatically learns optimal default split directions without manual imputation."

Follow-ups to expect

What is MICE (Multivariate Imputation by Chained Equations)? A series of chained regression models where each feature with missing values is modeled conditionally on all other features, cycling iteratively until imputed values stabilize.
How do you handle missing values during real-time production inference? If an upstream API returns NULL, use the pre-computed median value from the training feature store alongside the missing indicator flag, or let XGBoost route the missing value down its learned default branch.

Check yourself

Question 1 of 3

Which missing data mechanism occurs when the probability of a value being missing depends directly on the unobserved missing value itself (e.g. high-income individuals refusing to report income)?

More in Classical ML

See all →

Bias–Variance Tradeoff4 min Overfitting vs Underfitting3 min Linear Regression4 min