Outlier & Anomaly Detection

Detecting anomalous data points that distort statistical metrics or represent fraud, intrusions, and system failures.

🟡 intermediate5 min readdata

Outlier and Anomaly Detection identifies observations that deviate significantly from expected normal data distributions. Statistical methods use Z-score thresholds or Interquartile Range (IQR bounds: Q1 - 1.5·IQR, Q3 + 1.5·IQR). Unsupervised ML algorithms include Isolation Forest (isolating anomalies via random splits), One-Class SVM (fitting tight decision boundaries around normal points), Local Outlier Factor (LOF), and Autoencoder reconstruction error thresholds.

Outlier Detection vs Novelty Detection

Outlier Detection (Unsupervised): Training data contains clean inliers AND undetected noise outliers. Algorithm must identify anomalies within the training set itself.
Novelty Detection (Semi-Supervised): Training set contains strictly clean normal data. Algorithm learns normal baseline boundaries to flag unseen new anomalous events during inference.

       STATISTICAL (IQR)                    ISOLATION FOREST                       AUTOENCODER
   Outlier if x < Q1 - 1.5·IQR          Short Path Length h(x) -> Anomaly      High Reconstruction Error
     or x > Q3 + 1.5·IQR                Long Path Length  h(x) -> Normal          ||x - x̂||² > Threshold

Detection Toolkit Comparison

Algorithm	Method Class	Key Parameter	Best Used For
IQR Rule	Univariate Statistical	$Q1 - 1.5 \cdot \text{IQR}, Q3 + 1.5 \cdot \text{IQR}$	Tabular numeric features
Z-score / Modified Z-score	Gaussian Statistical	$	Z
Isolation Forest (iForest)	Tree-based Partitioning	`contamination` (expected % outliers)	High-dimensional tabular data
One-Class SVM	Kernel Boundary	`nu` ($\nu \in (0, 1]$ bounds outlier fraction)	Complex non-linear boundaries
Local Outlier Factor (LOF)	Local Density	`n_neighbors` (Compares local density)	Variable-density spatial clusters
Autoencoder MSE	Neural Reconstruction	Reconstruction loss threshold	Images, Audio, Time-Series

Isolation Forest Mechanics

Isolation Forest builds an ensemble of random decision trees (iTrees) by randomly selecting a feature and a random split value:

  Normal Point (Inlier)                       Anomalous Point (Outlier)
  Deep Tree Path Length h(x) = 14            Short Tree Path Length h(x) = 2
  (Takes many splits to isolate)              (Isolated immediately near root!)

Anomaly score $s(x, n)$ for sample $x$ over $N$ trees:

$$s(x, n) = 2^{-\frac{\mathbb{E}[h(x)]}{c(n)}}$$

$\mathbb{E}[h(x)]$: Average path length of $x$ across all trees.
$c(n)$: Average path length of unsuccessful search in a Binary Search Tree.
Score $s \to 1.0$: Definite anomaly.
Score $s < 0.5$: Normal inlier.

Say this out loud

"Outlier detection identifies anomalous data points. For univariate data, we use robust IQR bounds (Q1 - 1.5·IQR to Q3 + 1.5·IQR). For high-dimensional tabular data, Isolation Forest is the industry standard—it isolates anomalies via random decision tree splits, identifying outliers by short average path lengths h(x). For complex vision or time-series data, Autoencoders flag anomalies via reconstruction MSE spikes."

Follow-ups to expect

Why is standard Z-score problematic for outlier detection? Z-score $Z = (x - \mu)/\sigma$ uses mean $\mu$ and standard deviation $\sigma$. Extreme outliers inflate $\mu$ and $\sigma$ heavily, masking true outliers. Use Modified Z-score with Median and Median Absolute Deviation (MAD).
What is MAD (Median Absolute Deviation)? $\text{MAD} = \text{median}(|x_i - \text{median}(x)|)$. A robust non-parametric measure of statistical dispersion unaffected by extreme outliers.

Check yourself

Question 1 of 3

Why is Isolation Forest faster and more effective for high-dimensional anomaly detection than distance-based outlier methods?

More in Classical ML

See all →

Bias–Variance Tradeoff4 min Overfitting vs Underfitting3 min Linear Regression4 min