Bayesian vs Frequentist

Two Philosophical Worldviews

┌───────────────────────────────────────┬───────────────────────────────────────┐
│        FREQUENTIST APPROACH           │           BAYESIAN APPROACH           │
├───────────────────────────────────────┼───────────────────────────────────────┤
│ Parameter theta is a FIXED CONSTANT.  │ Parameter theta is a RANDOM VARIABLE  │
│ Data is repeatable random sampling.   │ with a probability distribution.      │
│ Output: P(Data | H0) (p-value)        │ Output: P(Parameter | Data) (Posterior│
│ Uses long-run sampling frequency.     │ Combines Prior belief + New Data.     │
└───────────────────────────────────────┴───────────────────────────────────────┘

Side-by-Side Comparison

1. Probability Definition: Frequentist: Long-run relative frequency of repeatable events as N -> infinity. Bayesian: Degree of belief or certainty given current information.

2. Priors: Frequentist: No priors allowed. Data must speak entirely for itself. Bayesian: Explicit Prior distribution P(theta) incorporating domain knowledge.

3. Interval Interpretation: Frequentist 95% Confidence Interval: 95% of intervals generated by this procedure will cover the fixed parameter. Bayesian 95% Credible Interval: There is a 95% probability that the parameter lies within this interval given observed data.

4. Peeking at Data: Frequentist: Peeking inflates Type I error rate, requiring strict sample size pre-commitments. Bayesian: Updating posteriors continuously as data arrives is mathematically valid.

When to Use Which?

Use Frequentist Methods When:

1. Regulatory standards require standard p-value compliance (FDA clinical trials, legal filings).
2. You want zero subjective prior assumptions to influence the result.
3. Computation must be ultra-fast and simple (t-tests, ANOVA).

Use Bayesian Methods When:

1. You have strong domain prior knowledge (historical conversion rates, physics bounds).
2. You are making real-time decisions under uncertainty (Multi-Armed Bandits, Thompson Sampling).
3. Stakeholders need direct probabilistic answers ("What is the chance variant B increases revenue by at least 2%?").

Say this out loud

Frequentists view parameters as fixed constants and measure long-run sampling frequency via p-values. Bayesians view parameters as random variables, updating prior distributions with new data to compute posterior probabilities. Bayesian credible intervals allow direct statements like 'there is a 95% chance the parameter is in this range', making results intuitive for business decisions.

Follow-ups to expect

• What is a Bayes Factor? The ratio of the marginal likelihood of two competing hypotheses: BF = P(Data | H1) / P(Data | H0). A Bayes Factor > 10 provides strong evidence for hypothesis H1 over H0.
• Why isn't everything Bayesian? Historical computational limits made calculating high-dimensional posteriors impossible before modern computers and MCMC sampling algorithms like PyMC and Stan.