Active Learning

What is Active Learning?

In traditional passive learning, human annotators label a random batch of 100,000 samples, and the model trains on whatever data it receives.

However, labeling 80,000 easy, obvious samples adds zero new knowledge to the model!

Active Learning flips the pipeline:

The model interactively inspects a massive unlabelled data pool and selectively asks a human expert (Oracle) to label only the most informative, uncertain samples:

  Unlabelled Pool (100,000 Samples) ──► [ MODEL ] ──► Selects Top 100 Most Uncertain Samples!
                                                             │
                                                             ▼
  Human Annotator Labels 100 Samples ◄───────────────────────┘
        │
        ▼
  Retrain Model ──► Repeat Loop! (High Accuracy using 90% fewer labels!)

Three Active Learning Scenarios

1. Pool-Based Sampling (Standard): Maintains a static pool of unlabelled data and evaluates informative scores across all candidates to select the top batch.
2. Stream-Based Selective Sampling: Processes data one sample at a time in real-time, deciding whether to ask for a human label or skip.
3. Membership Query Synthesis: Generates synthetic new data points where model uncertainty is highest and sends them to humans for labeling.

Popular Query Strategies

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. UNCERTAINTY SAMPLING  │ 2. QUERY BY COMMITTEE    │ 3. EXPECTED MODEL CHANGE │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Queries samples where the│ Trains an ensemble committee│ Queries samples that  │
│ model is most confused.  │ of models; queries points│ cause the largest gradient│
│ Entropy, Least Confidence│ with max disagreement.   │ weight update if labeled.│
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

1. Uncertainty Sampling Metrics

• Least Confidence: Selects sample $x$ where top predicted class probability is lowest:

$$x_{\text{LC}}^* = \arg\min_x \max_y P(y \mid x)$$

• Margin Sampling: Selects sample $x$ with the smallest difference between its top 2 predicted class probabilities:

$$x_{\text{Margin}}^* = \arg\min_x \left( P(y_1^* \mid x) - P(y_2^* \mid x) \right)$$

• Entropy Sampling: Selects sample $x$ with maximum prediction entropy across all classes:

$$x_{\text{Entropy}}^* = \arg\max_x \left( -\sum_y P(y \mid x) \log P(y \mid x) \right)$$

2. Query By Committee (QBC)

Maintains a committee of diverse models (e.g. SVM, Random Forest, Neural Net).

Queries unlabelled samples where committee members disagree most on predicted class labels.

Cold Start and Sampling Bias Challenges

1. Cold Start Problem: If initial seed labeled dataset is too small, uncertainty estimates are completely uncalibrated, selecting useless noisy samples.
2. Sampling Bias: Focusing purely on uncertainty risks sampling outlier noise repeatedly. Use Core-Set Diversity Sampling to balance uncertainty with feature space coverage.

Say this out loud

Active Learning interactively queries human annotators to label only the most informative unlabelled samples. Query strategies include Uncertainty Sampling (Least Confidence, Margin, Entropy) and Query By Committee. Active Learning maximizes model accuracy while reducing human labeling costs by up to 90 percent.

Followups to expect

1. What is Core-Set Selection in Active Learning? A diversity strategy that selects a subset of unlabelled points such that every unlabelled sample lies within distance $\epsilon$ of a labeled point, ensuring full coverage of feature space.
2. What is Human-in-the-Loop (HITL) Machine Learning? Production systems combining Active Learning with human annotator verification interfaces to continuously update models as new edge cases arrive.