Teacher Forcing & Exposure Bias

What is Teacher Forcing?

When training sequence generation models (like machine translation or language models), the model predicts tokens step by step.

At step $t+1$, what should the model receive as its input?

1. Option A (No Teacher Forcing): Feed the model's own prediction from step $t$.
2. Option B (Teacher Forcing): Feed the true ground truth target token from step $t$, ignoring any mistake the model made.

  WITHOUT TEACHER FORCING (Slow & Unstable Training):
  Step 1: Input "The" ──► Model Predicts "dog" (WRONG! True was "cat")
  Step 2: Input "dog" ──► Model Predicts "barks" ──► COMPOUNDING ERRORS! Whole sentence ruined!

  WITH TEACHER FORCING (Fast & Stable Training):
  Step 1: Input "The" ──► Model Predicts "dog" (WRONG! True was "cat")
  Step 2: Force Input "cat" (True target!) ──► Model Predicts "sat" ──► STABLE LEARNING!

Teacher Forcing acts like a teacher guiding a student playing piano. When the student strikes a wrong key, the teacher immediately points to the correct next key so the student learns the rest of the song properly.

Why We Use Teacher Forcing

1. Fast Convergence: Prevents early mistakes from derailing the entire sequence, allowing backpropagation to learn correct next step predictions across all positions in parallel.
2. Parallel Training in Transformers: Enables Transformers to process an entire target sequence at once during training using Causal Masking, running matrix operations across all token steps simultaneously on GPUs.

The Downside: Exposure Bias

While Teacher Forcing makes training fast, it creates a gap between training and real world deployment called Exposure Bias.

  TRAINING ENVIRONMENT:    Model ALWAYS receives perfect ground truth input tokens.
  INFERENCE ENVIRONMENT:   Model NEVER receives ground truth tokens; must feed its own past outputs!

During real world inference, if the model makes a small mistake at step 3, it is exposed to an unfamiliar state it never experienced during training.

The small mistake throws off step 4, causing errors to cascade exponentially until the output sequence degrades into gibberish.

How to Fix Exposure Bias

1. Scheduled Sampling (Bengio et al., 2015): Start training with 100 percent Teacher Forcing. As training progresses, gradually increase the probability of feeding the model's own predictions instead of ground truth tokens.
2. Reinforcement Learning Alignment (RLHF / PPO): Fine tune the model on full self generated output sequences scored by a reward model.

Say this out loud

Teacher Forcing feeds ground truth target tokens as inputs to step t+1 during training, preventing early mistakes from compounding and accelerating convergence. However, it creates Exposure Bias because the model never experiences its own mistakes during training, causing errors to cascade during real world inference. Scheduled Sampling fixes this by gradually blending self generated inputs during training.

Followups to expect

1. How does Causal Masking relate to Teacher Forcing in Transformers? In Transformers, causal masking hides future target tokens while allowing the model to compute loss across all sequence positions simultaneously using 100 percent Teacher Forcing in a single forward pass.
2. Is Teacher Forcing used in non sequential tasks? No, it is specifically designed for autoregressive sequence models where step t+1 depends directly on token t inputs.