Propensity Score Matching

The Selection Bias Challenge

Suppose you want to evaluate whether taking an online Data Science Bootcamp ($T = 1$) increases annual salary ($Y$).

In observational data, people who choose to enroll in a boot camp ($T = 1$) are not a random sample:

• They are younger, more motivated, and possess higher prior technical skills ($X$).

If you compare raw average salaries:

$$\text{Raw Difference} = \text{Mean}(Y \mid T=1) - \text{Mean}(Y \mid T=0)$$

The result will be heavily biased due to Selection Bias (confounding variables $X$).

  Observational Bias:
  Bootcamp Group (T=1): Younger, High Motivation, Tech Background  ──► High Salary
  Control Group (T=0):  Older, Moderate Motivation, Non-Tech      ──► Lower Salary
  (Did the bootcamp cause the salary increase, or did baseline user traits?)

Propensity Score Matching (PSM - Rosenbaum & Rubin, 1983) balances baseline covariates between groups, simulating a Randomized Controlled Trial.

What is a Propensity Score?

The Propensity Score $e(X)$ is the probability that a subject receives treatment ($T = 1$) given their baseline covariates $X$:

$$e(X) = P(T = 1 \mid X)$$

We estimate $e(X)$ using a binary Logistic Regression or XGBoost model trained to predict treatment assignment $T$ from features $X$.

┌──────────────────────────┬──────────────────────────┐
│ 1. DIMENSION REDUCTION   │ 2. ROSENBAUM-RUBIN THEOREM
├──────────────────────────┼──────────────────────────┤
│ Collapses 50 high-dim    │ Conditioning on scalar   │
│ covariate features X into│ e(X) balances ALL 50     │
│ a single 1D scalar       │ baseline covariates      │
│ probability score e(X).  │ between groups!          │
└──────────────────────────┴──────────────────────────┘

Step-by-Step PSM Pipeline

  1. Train Logistic Regression model on features X to predict Treatment T.
  2. Output Propensity Score e(X_i) for every individual i in dataset.
  3. For each Treated Subject (T=1), find a Control Subject (T=0) with NEAR-IDENTICAL e(X).
  4. Check Covariate Balance (Standardized Mean Difference < 0.1).
  5. Compute Average Treatment Effect (ATT) on matched pairs!

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ MATCHING ALGORITHMS      │ NEAREST NEIGHBOR (k-NN)  │ CALIPER MATCHING         │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Pair treatment unit with │ Matches control unit with│ Matches only if score    │
│ control unit.            │ closest |e(X_i) - e(X_j)|│ distance is within threshold│
│                          │ score distance.          │ (e.g. caliper = 0.02).   │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

Core Assumptions for Valid Causal Inference

1. Unconfoundedness (Ignorability): All confounders influencing both treatment $T$ and outcome $Y$ are measured in $X$.
2. Common Support (Overlap): $0 < P(T = 1 \mid X) < 1$. Every subject must have a non-zero probability of receiving treatment or control.

Say this out loud

Propensity Score Matching balances treatment and control groups in observational studies. The propensity score e(X) = P(T=1 | X) collapses multi dimensional baseline confounders into a 1D scalar probability score. Matching treatment units with control units sharing identical propensity scores eliminates selection bias, simulating a randomized trial.

Followups to expect

1. What is Inverse Probability Treatment Weighting (IPTW)? Instead of 1-to-1 matching, IPTW reweights each sample by $w_i = \frac{T_i}{e(X_i)} + \frac{1 - T_i}{1 - e(X_i)}$, using the full dataset to compute Average Treatment Effect (ATE).
2. What is Double Robust Estimation? Combining a propensity score model with an outcome regression model. The causal estimate remains unbiased if EITHER the propensity model OR the outcome model is correctly specified.