Sampling from Huge Datasets
Extracting representative training samples from multi terabyte datasets using Reservoir, Stratified, and Importance Sampling.
The Scale Bottleneck
Modern data lakes store terabytes or petabytes of raw interaction logs.
Attempting to load petabyte scale datasets into GPU memory for model training is computationally impossible.
To train models efficiently, we use Data Sampling Strategies to extract smaller, statistically representative training subsets.
Petabyte Data Lake (100 Billion Rows) ──► [ STATISTICAL SAMPLING ] ──► Representative Sub-sample (1 Million Rows)
Key Sampling Algorithms
┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. STRATIFIED SAMPLING │ 2. RESERVOIR SAMPLING │ 3. IMPORTANCE SAMPLING │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Divides data into class │ Uniformly samples fixed │ Samples instances │
│ strata first, sampling │ size K elements from a │ proportional to their │
│ proportionally to preserve│ streaming data pipeline │ informational value or │
│ exact class ratios! │ in a SINGLE pass. │ difficulty. │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘
1. Stratified Sampling (Preserving Class Balances)
When working with imbalanced classes (for example 99.9% legitimate transactions and 0.1% fraud), simple random sampling might extract a subset with zero fraud examples!
Stratified Sampling splits data into class subgroups (strata) first, then draws samples proportionally from each class to preserve exact population ratios.
2. Reservoir Sampling (Streaming Data Streams)
Suppose data arrives as a continuous stream of unknown total length $N$. How do you extract a uniform random sample of size $K$ without storing all $N$ items?
Algorithm:
- Keep the first $K$ items in an array (the Reservoir).
- For each incoming item $i$ (where $i > K$), generate a random integer $j$ between $1$ and $i$.
- If $j \le K$, replace item $j$ in the Reservoir with the new incoming item.
Every item in the stream receives an exact equal probability $\frac{K}{N}$ of being included in the final sample!
3. Importance Sampling (Prioritizing High Loss Cases)
Instead of uniform sampling, draw samples proportional to their loss or difficulty:
- High loss samples (hard examples) are sampled more frequently.
- Low loss samples (obvious easy cases) are sampled less frequently.
- Adjust loss weights using inverse probability weights to prevent statistical bias.
Say this out loud
Sampling from huge datasets extracts representative subsets for efficient model training. Stratified sampling preserves exact class proportions on imbalanced datasets. Reservoir sampling extracts uniform random samples from streaming data in a single pass without storing the full stream. Importance sampling prioritizes informative or high loss samples to accelerate training.
Followups to expect
- What is Downsampling vs Upsampling? Downsampling reduces majority class samples to match minority class counts. Upsampling duplicates or synthetically generates (SMOTE) minority class samples to balance classes.
- How does distributed Spark sampling work? Spark partitions data across cluster worker nodes, executing parallel local sampling on each partition before aggregating sample results.
Check yourself
Why is simple random sampling dangerous when dealing with heavily imbalanced datasets like credit card fraud?